Thoughts on the New AI Picture Smart in JAWS 2024.

The March update of JAWS contains considerable new enhancements to the Picture Smart functions. In my view this is one of the very best feature enhancements within JAWS for some time. The system uses both Gemini and Chat GPT services so as to provide descriptions of images from web sites, files or perhaps a screen-shot of an application.
You don't need a special account to use it. You can just enjoy it.

I have been using this for a few weeks now. Here are some practical use cases I frequently work through.
1. I am a very heavy user of X/Twitter as it contains a great deal of content in which I am interested and keeps me up-to-date with events. Many posts on X contain images. I am easily able to focus on the image and get an excellent description in return.
It also works very well on social media sites where, for example, a user has scanned in a page from a TV listings magazine of long ago as the text is rendered correctly.

For anyone who has my JAWS scripts for X/Twitter, an update will be available shortly where you can press a keystroke to more easily obtain a description of an image relating to the post.
The script will identify if there is an image for the post and if so will scan it and present you with the results. If there is no image for that post, JAWS will let you know. If you don't have the scripts, you can still use the function of course by using the keystrokes listed here.
Here is a description of an image from X:
The image is a collage of four photos of the Swedish pop group ABBA.

The first photo shows the group in a recording studio, with Benny Andersson, Anni-Frid Lyngstad, Björn Ulvaeus, and Agnetha Fältskog all standing around
a microphone.

The second photo shows the group in a more candid moment, with Björn and Agnetha sitting on a couch and Benny and Anni-Frid standing behind them.

The third photo shows the group at a press conference, with all four members sitting at a table and answering questions from reporters.

The fourth photo shows the group at an awards ceremony, with all four members holding awards.
The Chat GPT output gives more descriptions of the outfits they are waring.

2. In order to prepare videos for upload to my wife's YouTube channel, I use a program called Handbreak. When the video is loaded into Handbreak, I can do a quick scan of the window. If it gives me a good description of the video then I know the visual appeal is of a high standard. I can also carry out the same action when the video is uploaded to YouTube and this gives me confidence since I know it has been uploaded correctly and is looking good.

Descriptions are like this:
The image shows an indoor setting with two individuals. On the left, there is a woman who appears to be smiling and has short, light-colored hair. She
is wearing a bright red top and is seated on a dark-colored couch. The woman has her head slightly tilted and is laughing or speaking. She is holding a
baby with both hands.

The baby, on the right side of the image, is lying on a cushion or support that is partially visible. The baby appears relaxed and is dressed in a white
onesie with various animal prints on it. The onesie has full sleeves, and the baby's left hand is resting on the onesie, showing only the fingers. The
baby has a pale complexion and eyes partially closed, possibly sleeping or resting comfortably.

3. We have a large library of pictures taken over the years and scanning these and obtaining descriptions brings back memories.

In the image, you see a person dressed elegantly in a red outfit with a tiara on their head, indicating a festive or formal occasion. This individual is
wearing jewelry like earrings, a necklace, and a bracelet, all showing a glittering appearance. They seem to be biting into a piece of red and white dessert,
possibly red velvet cake with cream cheese frosting or a similarly styled cupcake, given the size and how it's held.

In the background, you can see a table laid out with drinks, such as wine and champagne, and it seems there is a gathering or celebration taking place.
There's another person visible in the background, sitting at the table, looking towards the camera with a glass in front of them. The room has a warm and
joyous atmosphere, with a vibrant red wall that adds to the festive ambiance. The setting appears to be a home dining area, indicated by the presence of
bookshelves and domestic furnishings.

Useful tips:
By default, JAWS will just give you a brief description of the image. You need to activate the More Results link to obtain description from both services.
The Gemini summary is presented initially because the response time is faster and you may just want a brief picture description.

The list of keystrokes is:
JAWS Key+Space then P then:
C for control.
F for file, to be used in File Explorer as an example.
W for window.
S for screen.

It is important to note that when using this feature with social media sites, when the analysis is retrieved the Results Viewer does not always gain focus. This is a concern. You need to ALT+Tab over to it and possibly press Down Arrow afterwards for the description to read. It does not always happen but it frequently can.

Apart from Picture Smart, the function to list the running applications, JAWS Key+F10, has a new enhancement. Find the name of an application which is running and press Delete. The application is closed. A nice enhancement to that tool.