Use ChatGPT with your On Screen Content through Siri

Hi, Jacob here. Today I’m going to show you how you can use ChatGPT to interact with on-screen content on your iPad. I’m going to use it to find some recommendations for places to walk to in Vegas. I don’t have long, but I want to try and find a few key spots I can walk to from the hotel.

I’m browsing Maps here and there’s so much going on – it’s really hard to know what the best things are.

All I’m going to do is interact with Siri by typing – but you could talk as well, just by double-tapping the Home Bar indicator at the bottom. From here, I’m going to start off my Siri request with the phrase Ask ChatGPT, and then I can simply write a prompt to ChatGPT.

So I’m going to ask it to use the map here to find some places that are worth visiting if I haven’t got very long in Vegas. Then, when I submit that request – or if I’m talking to Siri, when I just finish talking – I’m going to get a little prompt coming up at the top of my screen, just like this.

It’s going to ask if I’m happy to share the content of my screen with ChatGPT. I have the option here to either share just what’s on the screen – the screenshot itself – or, if I’m working in a Pages document or an email, I can share the whole document that way if I wanted to.

I just need a screenshot here, and then ChatGPT will get to work using the context of what it can see on the iPad screen, as well as its large language model.

Here, you can see it’s given me a list of recommendations, which are definitely things I want to look into a bit more and explore.

But actually, I want to fine-tune this a little bit. I feel like, if I’m going to be in Vegas, I want to at least look at the Welcome to Vegas sign at one end of the Strip. So I’m going to refine my prompt by simply typing into the text box here that I want some ideas around that end of Vegas instead – some suggestions around the Welcome to Vegas sign.

Then, when I send it, ChatGPT’s remembered the context of the conversation – it’s remembered what we talked about before – and just like that, it’s given me a new revised list of places that I might like to visit.

And of course, I can carry on doing this to refine the suggestions more, or perhaps I can zoom into the map and send another screenshot – all of those things to help me make the best trip I can in Vegas.

And this ChatGPT integration works with anything you’ve got on your screen in any app on the iPad. It will give you a prompt every time to make sure you’re happy to share, but basically, in your ChatGPT request, you just have to say use what’s on the screen or use what we can see here and it will automatically start using that content for you.

And don’t forget – if you want lots more iPad QuickTips, jump onto my channel down below, and don’t forget to press subscribe. I’ll see you next time.