OPINION

Putting OpenAI tools like ChatGPT to the test

Share:

ChatGPT can write brand copy in seconds, but how good is the output? Without goes up against the AI chatbot…

Everywhere you scroll, every link you open, every newsletter you read (hi!) seems to lead to a discussion about ChatGPT, the chatbot launched by OpenAI in November 2022. If you haven’t used it yet, your teams definitely have. From a brand storytelling perspective, what users prize the most is its ability to respond to a command – “write a brand story for the homepage of a healthy, fast food chain” – with well-written copy in seconds.

At Without, we believe words matter for two reasons. First, design that makes a difference is underpinned by a strategic concept that, initially, needs to be articulated to effect change. Second, good writing moves people.

So we decided to go up against ChatGPT with a team workshop over breakfast. To make this a like-for-like test, we set ChatGPT the same brief we’d given a copywriter the previous week, including a short set of guidelines on style and tone of voice. Here’s an edited transcript of our exchange (brace yourselves):


WO
Write 25 words about the swimming pool of a premium London health club, based on this information: “Mega 25m pool for performance training in tranquil space – a sea of blue, brass and bronze”. Use this tone and style: punchy sentences, edited lists, details (equipment/tech/size etc), one clear sentiment/achievement.

CHAT GPT 1. Dive into the sea of blue in the 25m Mega pool of the London health club. 2. Take a swim in the tranquil space and feel calm and relaxed.3. Enjoy the luxurious feel of the brass and bronze accents.4. Challenge yourself with performance training in the large pool.5. Get fit in the giant aquatic space […].

[this continued for a further 13 bullet points and 284 words].

 

WO What’s with the 18 bullet points? Just one paragraph, please. And remember: no more than 25 words.

CHAT GPT Dive into the sea of blue in the 25m Mega pool of the London health club and take a swim in the tranquil space to feel calm and relaxed. Enjoy the luxurious feel of the brass and bronze accents as you challenge yourself with performance training in the large pool […].

[…and so on; basically its first answer without the numbered list]

WO Crikey, that’s long! I did ask for a maximum of 25 words, please..

CHAT GPT Dive into the 25m Mega pool and feel calm and relaxed. Enjoy the brass and bronze accents and challenge yourself with performance training. Feel proud of your achievements and invigorated after swimming […]

[75 words, five minutes and many repeated requests for 25 words later…]:

CHAT GPT Dive 25m Mega pool, feel calm. Brass, bronze accents, challenge yourself. Proud, invigorated after swimming. Push, take a refreshing dip. Rejoice, take a break. Conquer, rejuvenate. Make a splash in vast pool.

WO Could you pretend to be human and write like they would?

CHAT GPT Dive into the 25m Mega pool and feel a sense of tranquillity. Enjoy the luxurious feel of brass and bronze accents as you challenge yourself. Accomplish your goals with the size and distance of the pool. Experience an energising swim in the large pool and bask in the calming blue hues. Take a break from the hustle and bustle of London and rejuvenate in the tranquil atmosphere. Conquer your performance training and make a splash in the vast pool. 

WO Now you’ve made it too long again: no more than 25 words. Just choose the most important points, if that helps?

CHAT GPT Dive 25m Mega pool, feel calm. Brass, bronze accents, challenge yourself. Proud, invigorated after swimming. Take a refreshing dip. Rejoice, take a break. Conquer, rejuvenate. Make a splash in vast pool.

We gave up. Our copywriter took less time to write her version the previous week, which included going back to the client to ask for more information – as a swimmer, she wanted to know about lanes:

Where training meets tranquillity. A 25m pool with 4 lanes designed for performance, to the backdrop of a calming sea of blue, brass and bronze.

There is no doubt that ChatGPT is a useful tool to speed up the technical labour of writing. For SEO-driven copy, it’s perfect: bots chatting to bots. When the aim is simply instructive or informative, and you feed it with all the details, it works a treat.

Sign up for insight, interviews and news from Out There.