Listen, I’m a pretty anxious fella. I’m the kind of guy who wakes his wife up at 2 a.m. to ask if she thinks we should stock up on dried beans and water just in case, you know, “we finally get hit by the big one.”
So when I first started reading about ChatGPT and its potential for making writers like me completely redundant, I got a little bit nervous.
But there comes a time when we must face our fears, and that time came for me at 10:17 a.m. on a Tuesday.
That’s when Clariant Creative Agency’s content team was tasked with trying out ChatGPT to learn more about how it worked, how it might integrate with our projects, and what benefits it might offer our clients.
Here’s what we discovered (you can skip to our results if you’re already familiar with how ChatGPT works).
What Is ChatGPT?
If your phone or email has ever tried to finish a sentence for you as you’re writing it, you’ve experienced a “language model” in action. Language models use probabilities to predict the next word in a sequence. So, if you start writing a text with the words “I love …,” the language model in your text messaging app will analyze what you’ve written in the past to predict how you might want to complete your sentence, whether it's with, “you” or, “doom-shopping for dried pantry staples.”
In a sense, this is what ChatGPT does: It’s an incredibly advanced language model developed by OpenAI.
But instead of analyzing the text messages of a single person, ChatGPT has analyzed … well, just about the entirety of the internet up until 2021.
It’s genuinely difficult to convey the magnitude of the input ChatGPT has consumed. The tool works by drawing on this vast database of input to provide responses to queries from users.
This means that as ChatGPT continues to scrape more information from the internet – and as more people engage with and provide feedback to the tool – it will become smarter and smarter. Noooo!!! 😲
All of this leads to a potentially incredibly powerful instrument that can respond to user direction, understand prompts, and create texts all on its own. For marketers, this is a big moment. Ask ChatGPT to write you a blog, and it will produce one in seconds. The same goes for social media content, emails, or even poetry.
If you use content to market your business, you can get that content from ChatGPT in the blink of an eye.
The only question is, should you?
Content Marketing and ChatGPT: Potential Use Cases
At Clariant Creative, we spend all day every day creating high-quality content for our clients. To understand whether there’s a role for ChatGPT in our content production processes, our content team ran a series of seven experiments using the tool in various ways. Here’s what we learned.
Test 1: Can ChatGPT translate raw interview transcripts into blog content?
Whenever possible, it’s a good idea to interview a subject matter expert before you begin writing content. We always record our interviews, but those raw transcripts typically take some work to polish up. Could Chat GPT help?
Results: Not great!
ChatGPT wasn’t good at understanding context. In the example above, the first paragraph represents a raw transcript from an interview we conducted, discussing data collection in hospitals. The second section in the example represents ChatGPT’s attempt to write a blog post from that transcript. You can see that the interviewee does mention standardizing care procedures. However, ChatGPT misunderstood the context behind that comment and instead accidentally introduced “standardized operating procedures” as a form of data collection – which is incorrect.
We suspect that ChatGPT struggled here because, due to the conversational nature of the interview, the information wasn’t organized clearly enough for the tool to properly contextualize it, thus leading to the error.
Test 2: Can ChatGPT write a blog from scratch?
Blogging is the meat and potatoes of many a content strategy. Could ChatGPT help lighten this lift, with just a simple prompt?
Results: Better, but bland
Here ChatGPT did a nice job of recognizing the specific organizational directions we included with our prompt (e.g., “include an introduction, three sections,” etc.). However, the content itself is … meh. There’s no unique point of view or thought leadership, and the content is not particularly compelling to read. Ultimately, all ChatGPT can draw from is the content that’s already out there – which means there’s nothing here that would make ChatGPT’s content stand out from any other competitor’s blog post about the same topic.
So while our result with this experiment isn’t awful, it’s certainly not what our team would consider high-quality output.
Test 3: Can ChatGPT write a blog from an outline?
Our previous experiments were showing us that the quality of ChatGPT’s output relies heavily on how we structure our prompts. So we wanted to see if providing ChatGPT with a more detailed outline would result in better content.
It did not.
Results: A little too literal
As you can see from the first image above, we provided extensive guidance in our prompt. But interestingly, ChatGPT completely disregarded our word count request. And whereas we were hoping the tool would translate our bullet point prompts into full concepts, that didn’t happen. Rather than put flesh on the bones of our outline, ChatGPT simply turned our bullet points into sentences and called it a day – which was, frankly, disappointing.
Test 4: Can ChatGPT write an outline from a high-level idea?
What if we leaned into ChatGPT’s organizational abilities and see how well it can structure a post (without having to actually write the whole thing)?
Results: Pretty good!
I was honestly a little impressed by this one. ChatGPT did a nice job building an outline that connects the dots between emotional intelligence and the work that help desk agents do.
That said, the outline is also a bit repetitive and lacks creativity. When you compare the Introduction and Section 1, you can see both sections follow the same basic format of “Explain > Provide Examples > Discuss.”
It’s not that any of this is bad necessarily, just that it underlines ChatGPT’s tendency to produce formulaic results … which could bore readers after a while.
Test 5: Can ChatGPT brainstorm blog topics and content ideas?
Sometimes coming up with an angle to write about can be the hardest part of content marketing. Might there be a role here for ChatGPT, given its enormous warehouse of insight into what people are searching for and what other companies are writing about?
Results: Now we’re talking!
As you can see by our prompts, this one took a little refining. But once ChatGPT understood what I was after, it produced some impressive results instantaneously.
I find that to brainstorm well, I have to silence my inner censor that wants to judge my ideas, and instead simply come up with as many ideas as possible. ChatGPT plays right into that and offers a surprising number of angles.
This breadth of results may be exactly what you need to find a fresh take on the topics you’re writing about. I see real potential here for ChatGPT to become a useful tool for content topic brainstorming.
Test 6: Can ChatGPT write an introductory paragraph?
Getting started is another common challenge content marketers face. Staring at a blank page on a screen, writing that first sentence can be a daunting experience – even for experienced writers. What if we used ChatGPT to help us get a piece of content going?
Results: A helpful tool for writer’s block
We’re learning that one of ChatGPT’s little quirks is a tendency to reword the prompt in its output – and you can definitely see that happening here. Our prompt refers to dental hygienists, and ChatGPT’s oh-so-subtle solution is to open with, “As dental hygienists, we have always known …”
But setting that aside, we saw some good potential here, too. In our experience, it’s often easier to edit something than it is to write something from scratch. So if you’re struggling with writing a particular section of your content – whether it’s an introduction, transition sentence, or a conclusion – you might try letting ChatGPT get something on screen that you can react to – even if it’s not perfect right out the gate. That little push might be exactly what you need.
Test 7: Can ChatGPT write a personal communication, like a letter or an email?
So we’ve seen how ChatGPT does with traditional forms of content marketing, such as blog posts. But we also wanted to test other formats, as well.
Results: A good start
If there’s one thing ChatGPT understands, it’s form. Tell it to write a haiku, blog, email, or letter, and it immediately produces something that fits the parameters of the genre you’re writing. (Just watch details such as word count requirements, as we noted earlier.)
While the output in our experiment here still uses the kind of generic language ChatGPT so often seems to lean on, this draft is a surprisingly good start. With enough rewriting and polish, this could well serve its purpose and potentially save the writer a fair bit of time.
The Big Reveal: Key Takeaways About ChatGPT and Content Marketing
I hope you’ve found this post interesting and useful so far. If you have, then you may be surprised to learn that this entire article was written by ChatGPT.
Ha ha – kidding (of course)! But how wild would that have been? You’d be like, “Oh my gosh,” and I’d be like, “Right?” and we’d both go and cry for a while.
The truth is that while ChatGPT is an incredible tool, there are many ways in which it’s still limited. As you consider how (or even if) you want to use it in your own content marketing, here are some takeaways to keep in mind:
- We may be on the cusp of a flood of AI-generated content. If you want to stand out, it’s more important than ever to be original, interesting, and useful.
- ChatGPT works fast, but it’s prone to cliché, formulaic content, and factual errors. Anything it produces must be reviewed (and probably rewritten).
- ChatGPT was trained on preexisting content. It can’t offer new insight, thought leadership, or unique expertise. (That has to come from you.)
- Google has been at this for a long While the search engine has said it isn’t against all AI-generated content, companies who use ChatGPT to publish reams of unedited, unreliable, generic content may find themselves losing domain authority and struggling to rank well once the algorithm catches up.
Our biggest takeaway? If you want your content to rank well and stand out from your competitors, you need to pair great writing with a smart content strategy.
If you’re not sure where to begin — and if the whole idea of this makes you anxious enough to wake your own partner at 2 a.m., well then, we’d be happy to help.
