There seems to be a stark divide in the content community in response to the soaring success of ChatGPT.
Some say that ChatGPT isn’t at all viable for use. Others are quick to point out that this technology will replace writers. Well, if there’s any truth to these viewpoints, it sits at the intersection. Neither is ChatGPT useless nor is it a replacement.
It’s a great tool (much like the host of AI tools out there) that can help with writing inspiration, testing ideas, and even seeking novel ideas. I’ve been using GPT-4 (ChatGPT Plus) for the past two days now, and I don’t think I’ve ever had a better writing companion. It’s that good! One might even think that it’s conscious, but that’s a debate for another day – although Sam Altman will outright deny the claim. 🙂 It’s still a question as to how he knows what “conscious” means and if AI has achieved it.
Today, we’re going to look at a very specific long-form use case of how-to guides against the growing prominence of AI Writers.
Are AI Writers Good at Writing How-To Guides?
Contrary to the popular belief, how-to guides are one of the most difficult forms of content to conceptualize for two reasons:
They have to be exceptionally simple in their articulation. After all, easy reading is damn hard writing.
They must be factually relevant and recent to help the user navigate the path they’re seeking answers to. The need for “recency” can vary based on the subject under question. For example, there’s hardly any relationship between time and playing chess.
But, here’s a concern – Is “how to play chess” similar in complexity to “how can CIOs harness the power of quantum computing for breakthrough innovations in B2B applications”?
You’d notice that the AI writing segment focuses on very generic use cases when it comes to marketing or demo videos. They would exhibit AI answering queries that are too broad to befit any robust content strategy.
In fact, the tools are inherently much more capable when it comes to elaborating the steps to achieve something. That’s precisely the hypothesis we laid out for further investigating the viability of AI writers for long-form B2B content – especially how-to guides.
The Results from Our Experimentation
We tested the top AI writers and AI writing assistants to help us craft how-to articles like:
How to leverage metaverse technologies for B2B collaboration?
How to validate and review data annotations?
How to leverage swarm intelligence and distributed robotics for industrial applications?
We limited the article length to 500-600 word range. The results were satisfactory at best.
It took more than 3 hours to pen factually and conceptually correct articles. Only journalistic & opinionated articles took more time.
Highly technical guides were largely difficult to pen because AI would often hallucinate. So, the human writer had to intervene after every few words to check for fallacies and make the necessary improvements.
Should you not leverage the capabilities of AI tools? You definitely should! And AI is going to evolve, so you can expect it to comprehensively help with moderately technical articles, if not highly technical ones.
But as someone leading a B2B enterprise, you must also understand that there’s a lot of data associated with your projects, functions, workflows, etc., at your disposal. If you can inspire your narratives based on this data, you’d be looking at a thought leadership piece that’s authentic and, most importantly, credible for users.
This would be in alignment with Google’s E-E-A-T (Experience, Expertise, Authoritativeness, and Trustworthiness) principles and provide much more impetus for the articles to rank on the SERP against websites with high domain authority.
More to come in this series around AI writers’ viability for other content types. Stay tuned!
Apart from the above, what more would you like to know? Let us know your thoughts and opinions in the comments below.
“We’re Living Through the Boring Apocalypse,” reads a headline from a 2021 New York Times opinion piece by the famous psychologist Adam Grant. This article was in response to the repeated blasting of emergencies during COVID-19 and how that was making people pay less attention by the day.
I’m starting to think that this applies to almost everything – especially B2B content marketing. The attention spans have dwindled, and the constant reiteration of uncharacteristic content justifies that.
So, it makes sense why a lot of focus is lent to “thought leadership content” — perhaps the most credible, unique, innovative, insightful, expressive, and reliable source of information out there.
Thought leadership content:
Differentiates the brand
Brings it out of the competitive fuzziness
Allows it to define the relationship with the space it’s serving
Positions it as an authority – a “knowledge” leader in the respective discipline
However, this meaning has dwindled a bit in the past few years. The phrase “Thought Leadership” is thrown around so often in the social space today that it seems as if it’s losing its purpose – because people often don’t understand what it entails.
Now that AI writers have graced the space, no wonder they’re being looked at as the means to amp up the thought leader content pipelines. Who knows if this space might become a commodity that can be completely outsourced to AI in the forthcoming months?
But should we be doing that? This needs thinking and rethinking because the answer has to be sourced by dismantling the elements that lay the foundation for thought leadership.
That’s what we’re going to do here.
The Source of Content
Where is the content coming from?
A great thing about thought leadership is that it allows businesses to forge a relationship between their capability and the needs of the respective discipline they’re serving.
This relationship depends on a host of things:
How industry experts perceive the market and want to position their offering
What is the data to support the opinions of the industry leaders
What is the data to support that a problem is being solved
What are objections, if any, to the unique wisdom being laid out
How is the stakeholder network being leveraged
These facets have one thing in common – they’re driven by human experience and insight. So, ultimately, the source of content is human. This can take the shape of:
Interviewing the in-house experts
Sharing case studies that evidence the brand’s capability
Sharing data and insights that are based on industry research
Outlining the mistakes that the brands have committed (and how they’ve overcome those challenges)
Weaving in what the network is talking about
Outlining what makes the unique product/service befit the need of the target audience
Summarizing the insights that have been drawn from the customers
That’s just a brief list of options. The point is that thought leadership content is sourced from people for people.
First off, there’s no uniqueness in the content because AI is essentially determining the best course of action based on the data at its disposal. For example, ChatGPT’s knowledge is limited to 2021.
Second, everything that’s being laid out in terms of “experience” is a fabricated story – which again is not authentic, unique, and original.
Third, there is no understanding of space because AI is not perceptive like humans. It can’t pull specific case studies from the brand-specific stories – it can mine some data around the subject in general and come up with a discourse that is surface-level at best.
Fourth, there’s no validation of the insights and industry expertise being shared. Everything laid out is likely a result of AI hallucinations.
So, even if you manage to write a blog using AI, you’ve probably edited it numerous times to make sure that it’s brand-specific, insightful, and relevant. But at what cost? Because it still wouldn’t be as good as something you’ve written from scratch. That brings us to our second facet – writing.
The Narration
By now, we understand that AI is not to be referred to for the information that’s going to be laid across the length of the article. But what about writing? Considering we have all the relevant information and a profound understanding of the direction we need to take, how about using AI by giving it prompts and providing cues to generate content at speed? Sounds reasonable, isn’t it?
We experimented with this. But our research into the viability of AI writing tools for B2B tech content revealed that:
On average, it took around 3 hours to write a thought leadership article of 600 words (after the research had already been done).
More than 70% of the content for these articles was written by humans as a part of giving the AI definite directions and intervening wherever it seemed to hallucinate.
Most of these articles were mediocre at best when compared to what human writers had been delivering.
The anecdote here? The AI writers aren’t making thought leadership content better – and they’re not easing the burden on writers, either. If anything, they’re diluting the narrative by making it less original.
And there’s a big reason for the same. Thought leadership content isn’t actually a content type; it’s a practice — a form of narration that a business espouses to stand out and reflect upon its authority across a discipline. AI writers haven’t been programmed this way.
Broadly, there are two categories of AI writers:
AI Writing Assistants – Those that provide writers a canvas to pen and use AI to complete the sentences or suggest ideas
Long-form AI Writers – Those that write a complete article on the go based on a keyword or title that has been provided as an input
What we experimented with were AI writing assistants – predominantly because we wanted more control over the narrative. The second category, i.e., long-form AI writers, are better suited for churning out extremely basic, SEO-focused content. The usefulness of such content is a debate for another time.
All in all, across both categories, the problem of the sameness of content is profound – which goes absolutely against what thought leadership stands for – uniqueness and credibility.
A Summary of the Discrepancies
Hallucination
AI can mix up facts, statistics, and even concepts. But it sounds lexically intelligent and convincing, so it’s challenging to discern whether something outlined is right. This requires subject matter expertise and constant monitoring.
Deviation
Thought leadership content in the B2B tech space, in particular, has a narrow and concrete focus. Because you’re not controlling the output beyond the prompt given or the manual editing after the AI has written, there are high chances that it constantly deviates from what you want to say.
Recency
Up until now, it has been challenging for AI to outline something that’s recent. The dated data set doesn’t serve the purpose in a space where narratives need to be catered to the “present” concerns.
Authenticity
Well, carving out unique content is almost impossible because it’s all computation on AI’s part. The textual outputs are a result of excellent language correlation around a subject; however, there’s no personality or uniqueness.
Data Storytelling
The knowledge residing within your operational workflows and the successes that you’ve had with your clients is invaluable. It’s directly related to your business’s capabilities and how they serve the discipline in focus. AI cannot replicate this.
So, there you have it!
More to come in this series around AI writers’ viability for how-to guides and other content types. Stay tuned!
Apart from the above, what more would you like to know? Let us know your thoughts and opinions in the comments below.
Google Is Freaking Out About ChatGPT (The Verge, Jan 2023)
The ChatGPT AI Hype Cycle Is Peaking, but Even Tech Skeptics Don’t Expect a Bust (CNBC, Feb 2023)
The Inside Story of How ChatGPT Was Built From the People Who Made It (MIT Review, March 2023)
All these are headlines reflecting upon the fact that ChatGPT has had the best Public Beta of all time.
Every nook and corner of search engines and social media has been flooded with narratives – some hyping it even more, others talking about the possible improvements, and a few informing how it has taken human jobs already.
But while a lot of ChatGPT text-generation use cases and applications have surfaced over the past few months (thanks to excellent prompt engineering), there are big question marks around AI’s viability to create long-form content – especially for a B2B audience.
Is AI Any Good for Long-form B2B Tech Content? (Our Research Attempted to Answer This)
The argument for writing articles and blogs using AI goes back to Q1 2021 when GPT-3 AI tools were being rolled out one after another.
In October 2022 (before the launch of ChatGPT), we started testing the top GPT-3 tools to understand how well they do for B2B articles. We tested them across major technology themes (like cloud, data analytics, cybersecurity, etc.), and here’s what we observed:
AI writers contributed (in terms of text used for the final article) less than 50% for more than 55% of the articles.
Less than 30% of the text was contributed by AI for 20% of the articles that were highly technical.
Surprisingly, listicles weren’t easy to create with AI – with 600 words articles taking more than 2 hours of continuous work on average. In fact, How-to guides and journalistic articles took more than 3 hours.
Of course, these results can be a bit biased in a sense that:
We focused on creating high-quality consumer-facing content based on our decade-long experience writing for the B2B audience. And so, we scrutinized and edited the AI outputs significantly before considering them.
We tested both technical and highly technical articles, which weren’t presumably in the purview of AI with respect to factual and conceptual accuracy.
But that’s the thing. When we’re talking about consumer-facing content, there’s no room or excuse for compromising the quality. And so, the question pops up – Why exactly are we using AI writers?
Is it because human writers aren’t adept? (Surely, that isn’t the case)
Is it to increase the frequency of publishing content?
Is it to lower the costs of content creation?
Is it to increase productivity?
The reasons could be manifold. But they often get obscure when riding the hype train. So, almost every answer will include the umbrella terms “productivity” and “cost.” And that’s understandable since these are integral to a business’s bottom line.
But then, what exactly does productivity entail here? How is it being defined? And how are the cost advantages being realized?
One explanation could be amping up a writer’s workflow – equipping them with an AI tool that reduces their cognitive load. So, with relatively lesser cognitive input, they would be able to create a similar-sounding output.
In fact, AI writers accrue several benefits for inspiring ideas, removing writer’s block, and helping with lo-fi content outlines. We’ve compiled the good things here.
But a lot of the presumable capabilities fall flat when it comes to creating long-form B2B tech content. Why’s that the case? Let’s discuss.
1. Original Content
Before getting into the granularities of B2B-esque content creation, let’s address the elephant in the room, shall we?
Although duplicate content (or plagiarism as it’s called in academic circles) isn’t always penalized by Google, it’s still an aspect of content creation that must be avoided. At the end of the day, search engines (especially Google) want original, helpful content to come up.
However, often, AI is very standardized in pushing out answers to your queries. Here’s an example. I asked ChatGPT – what is low-code development? But I did that three times – at different times. The answers, albeit quite relevant, overlap both structurally and linguistically.
All these explanations start with a definition, transition into the goals, approach the benefits, and outline the applications. From the looks of it, that’s an ideal, all-encompassing explanation – but only for the purpose of learning and not creating original content.
Notice the repeated keyphrases across definitions and the astonishingly similar sentence structure accommodating them. Even if you change the prompt to, let’s say – “explain” or “define” low-code development, the language used is similar.
These are just three queries from a single account. Imagine how many people will be asking the same questions. So, even if you structurally refine the content, you’re still leaving much to be desired when it comes to originality.
A possible and reasonable refutation to the same can be put forth in the name of prompt engineering. What if you give highly-specific, example-laden prompts and probe the AI until it gives you a quality output? Well, that’s surely opportune. However, that will also involve structural overlap in how the prompts are created. Why? Because the entire field is based on formulaic frameworks – much like you keep seeing on LinkedIn these days. So, it’s highly likely that the most relevant prompts are exceptionally similar.
2. Service/Capability-Oriented Writing
A good B2B content strategy supports a mix of informational and promotional content. The latter is concerned with reflecting upon a business’s capabilities to potential leads. This can be realized via:
Showing how the products or services can benefit the potential businesses. Case studies, for instance, help build awareness around how you can help and encourage businesses to move forward.
Showing how your product or service fits into a business’s current strategy. Most businesses, especially smaller ones, aren’t often looking to overhaul their entire strategies — they’re looking for incremental improvement and performance by integrating your product or service into what they’ve already put in place.
Elaborating on the differences between your product/service and the competitors’ — to help businesses identify the value of your offering over theirs. Comparison landing pages, for example, are a great way to show how your product or service stacks up against the competition in terms of the quality of your product, the value of your service, and how the business would benefit from a switch.
Putting out thought leadership content that drives business owners to think about their current practices, the industry as a whole, and your offering in relation to both.
Besides, a host of B2B businesses help implement products from global vendors. For example, IBM, ServiceNow, Salesforce, Microsoft, Amazon, SAP, and Google have product and service implementation partners (the B2B businesses) helping spread the word about their products and lowering the entry barrier to their adoption. Such businesses center their content around the capabilities of these vendors, and understandably so.
Why isn’t AI a good fit here?
For one, GPT-3 powered tools cannot scrape data from the web as per a business’s specific preferences. In general, they can control the subject-related output from a technical standpoint using certain parameters like temperature, top p, frequency penalty, presence penalty, and more. These parameters work to increase the creativity, originality, and recency of the outputs. As such, a lot of AI writing tools allow users to tweak the creativity levels of the text.
However, stringently controlling the output to talk accurately in terms of business offerings is a whole different ball game and certainly something that hasn’t been accomplished – at least until now. It’s even more challenging when we consider unique offerings from small businesses – a capability that the B2B SaaS landscape thrives on.
And this is also where the problem of emulating a brand’s voice comes to the fore. But can’t you have an editor refine that? Certainly! Humans working through AI outputs would only do good to the final outcome.
However, we’re talking about substantial edits in the case of promotional writing. And we’re giving more than the required control to the robot since the rough draft would be created by it. So, eventually, the output would still be inferior. This brings us to the question asked earlier – Why exactly are we using AI writers? Because this workflow certainly doesn’t sound productive.
A refutation to the brand’s voice facet could be AI tools that can help maintain consistency in the tone and voice – much like Grammarly Business which features a Tone Detector and Company Style Guide. With the help of these features, businesses can fixate their terminology and tone and align the content according to it. But then again, this needs substantial due diligence on the part of humans to set everything up. And the writing is manual.
3. Time-Bound Content
“Recency” is the hallmark of the B2B SaaS landscape. And why go far to understand that? The topic we’re addressing in this article serves as a great example.
Back in May 2022, Rob Toews wrote on Forbes that “a wave of billion-dollar language AI startups is coming.” Safe to say, it’s already here.
VC investments in Generative AI increased by 425% between 2020 and 2022. In 2023, this is the number one funding area. Out of 183 startups in Y Combinator’s first 2023 batch, 51 are AI startups, and 32 are explicitly into Generative AI.
As a result, AI and specifically GPT-3 is now powering apps for multiple use cases. These include, but aren’t limited to:
Workflow productivity
Content and copywriting
Image and video generation
Text to speech
Health monitoring
Virtual assistance
Programming
Such quick evolution of the space demands that the content that’s being pushed out:
Concretely outlines the “evolving” consumer pain points
Presents the product as a solution to those problems
Differentiates the brand from competitors
Evolves as the product landscape evolves
So, a good portion of such content is time-bound — i.e., it draws upon the recent assessment of the market, the experiences of CxOs in navigating the complexities of such market, the evaluation of the product itself, and the ongoing developments in the space. This differs from the B2C space, where products have a set description. This description remains independent from the next version of the product. For example, you wouldn’t have to tweak iPhone 14’s product description when iPhone 15 comes out.
In the B2B world, however, even the products or services that don’t serve a market as aggressive as AI entail a more rigorous process of iteration and refinement. As a result, the content that’s being put out needs to be time-bound.
Does that mean that every write-up should completely build on recency? Definitely Not! For example, HubSpot’s compilation of templates for sales, resumes, copywriting frameworks, etc., would work well for a long time.
However, it’s highly likely that a good portion of your content strategy is centered on weaving in recency to accommodate “relevance.” Even the most basic of articles – like XYZ trends about a [technology] in the [industry] space would warrant a B2B-centric focus on recency.
Why isn’t AI a good fit here?
One explanation is the dated data set that the AI is trained on. For example, ChatGPT inspires answers from its knowledge base, which is limited to 2021.
Even if the data corpus is updated in real-time, factual and conceptual discrepancies are likely to pop up. That might be due to the limited amount of data these tools have at their disposal regarding recent developments. Further, it’s just not possible to source statistics or facts. There’s a very high probability that the stats laid out by AI are fabricated and fictional.
Cuing back to our four-month-long research; throughout the process of testing GPT-3 AI writing tools, we observed that the articles involving the time element were almost impossible to create without successive interventions. These interventions were associated with:
Tweaking the direction that AI was taking
Manually including research and statistics to provide context
Ensuring that the text generated was factually on-point.
These articles ranged from exploring technological developments to how-to guides for explaining the best practices going forward.
On average, we could only let the AI write the next 9-10 words before we intervened.
4. Personality
“Computers have never been instruments of reason that can solve matters of human concern; they’re just apparatuses that structure human experience through a very particular, extremely powerful method of symbol manipulation.” — Ian Bogost, The Atlantic.
Structuring “human experience through a very particular, extremely powerful method of symbol manipulation” — this is perhaps the aptest and most concise way to sum up the personality argument.
Going back to the example shared earlier about low-code development, did you notice how standardized the response was? Now, such standardization still works well for definitions. But what if it encompasses every nook and corner of your narrative?
For one, it wouldn’t engage people; it wouldn’t draw them in. Second, it wouldn’t support the brand voice a business wants to realize. Most importantly, it wouldn’t do justice to the solution under consideration.
For example, I asked ChatGPT to create four introductions for a topic – each following a particular tone descriptor (Professional, Upbeat, Conversational, and Authoritative). While there are subtle distinctions in the sentence structure, they all read the same from an emotional standpoint. Conversational sounds a bit different, but it’s more casual than it’s conversational.
It’s still possible to sell a bad product through good marketing. But when the marketing’s bad, it doesn’t matter how good the product or service is.
In the B2B space, personality is even more critical. Your audience is most likely educated on the service/product they’re seeking. If not as well informed, they’re at least quite aware of what it takes to deliver on the promise and understand the level of expertise they’re seeking to navigate complex problems. In that light, the personality of your narrative and self-expression matters a lot. And this is reflected in the long-form content that builds the platform for a product or service to succeed.
The Bottom Line
While aesthetically pleasing, it’s one thing to integrate AI into your workflow and completely another to be persuaded by the guise of its grammatically sound articulation. No doubt, the lexical aptness gives AI writers remarkable fluency in their arguments. However, their storytelling capabilities are not to be mistaken as excellent.
Use them to increase the quality of your narrative, not to define it.
More to come in this series around AI content detection and whether AI writers are good for listicles and how-to guides.
Stay tuned!
Besides the above, what more would you like to know? Let us know your thoughts and opinions in the comments below.
Probabilistic determination of the next best possible word – that’s certainly the essence of AI writing tools powered by the famous language models GPT-3 and now GPT-3.5.
It’s all computation, and it’s reasonable when people point out – well, where’s the personality in writing? Where’s the technical nuance? These questions reflect legit objections. In fact, our months-long research outlined how:
AI writers contributed less than 30% (in terms of text used for the final write-up) for highly technical B2B articles.
The articles where AI writers contributed more than 70% were extremely basic, definition-oriented, and moderately or non-technical.
But why these questions in the first place?
Possibly because people are being led to believe that AI writers are the be-all and end-all. But if we closely monitor the LLM (large language model) space over the last three years, even the most prominent AI writing tools out there recommend refining content using human expertise.
Truth be told, AI writers aren’t exactly a replacement for humans. They are, instead, immensely viable for augmenting a writer’s capabilities. When we look at them from that perspective, we can outline the many benefits they can accrue for long-form content creation in the B2B space. And that’s what we’re going to do here.
For ease of illustration, we’ll use outputs from ChatGPT for this article.
1. Providing Writing Inspiration That Isn’t “Time” Bound
If there’s one thing that’s constant in the B2B space – it’s “change.” Technologies constantly evolve, vendors accommodate such evolutions, businesses invest in the new roll-outs, and the entire vertical goes through a metamorphosis.
The same is reflected in writing. Even the most generic of narratives want to weave in innovations and recent trends. It’s through those that they can truly reflect upon the “value” being added in the “present” context – via a service or a product.
But this doesn’t detract from the fact that AI can lower the entry barrier to basic business or technology-related ideas or perhaps add to the ideas that you’ve already laid out.
Let’s say you are creating a guide titled – “Best Practices to Create a Fintech Mobile App.” ChatGPT can come in handy to get the gist of the potential user pain points that must be considered.
For example, let’s ask ChatGPT to give us three user personas for a fintech mobile app.
The response is not bad. It provides potentially useful information that could fit our narrative. But since we aim to outline the pain points, let’s probe the response further to understand each persona’s challenges.
Not bad at all! We could now use these to substantiate what we already know and develop an example-laden, all-encompassing article around the best practices.
2. Helping Write Templatized Content Elements
Meta descriptions, SEO titles, email subject lines, image captions, confirmation messages – a host of content elements fall within the purview of frameworks. SEO titles and meta descriptions, for example, must adhere to a certain character limit, and it bodes well if they include the targeted keyword.
Let’s ask ChatGPT to create a title and meta description for the aforementioned article – “Best Practices to Create a Fintech Mobile App.”
Not the best, but it lays the groundwork. It can be refined more based on the article’s content.
Likewise, let’s use ChatGPT to create a catchy headline for this article.
Not bad! Can these be improved? Certainly! But again, these preliminary recommendations provide much to go about and save time.
Note: Understanding what you want to write about, what’s your business focus, and how you want to tell the story is critical before using AI to generate recommendations. Because this knowledge is what’s going to help discern if what you’re looking at is worthy of including in your process. Don’t be blindsided by the guise of immaculate articulation. It’s not that ideal – not more than your experience-led story.
3. Creating Lo-Fi Article Outlines
AI writers aren’t well-suited for consumer-facing B2B content. For one, the output can be factually and conceptually incorrect. Second, the content is exceptionally standardized – lacking personality, vision, and the ability to emulate a brand’s voice and tone.
But this utter standardization, i.e., lexical and structural exactness of GPT-powered textual output, makes AI writers well-suited for a basic content flow.
Let us call this flow a “low-fidelity content outline” or “lo-fi” outline.
Let us ask ChatGPT to create an outline for one of the blog titles that it previously suggested.
“5 Proven Strategies for Creating a Top-Performing Fintech Mobile App in 2023”
Continued…
The response is basic at best. Perhaps more probing would result in a more profound outcome. But we can still outline a few key things. For example:
Personalization and customization can create an opportunity to discuss SaaS capabilities like data analytics, business intelligence, rewards and loyalty programs, and more.
The content can be segregated across two major themes – design and technology. Design can cover UX/UI design and content design. Technology can cover security, integrations, analytics, and more.
All in all, a comprehensive content brief can come to life in relatively less time.
4. Taking Good to Excellent
All these capabilities suggest one thing – that AI writers can help human writers. And this can be realized in terms of:
Speeding up research
Speeding up writing
Including analogies and examples
Reducing cognitive fatigue
This can speed up the entire content generation workflow and open opportunities for scalability.
But the “quality” of the output would still depend on human competence.
More to come in this series around:
Things that AI’s Bad at
How can you detect AI content?
Why AI fails at writing thought leadership content? (and why you shouldn’t opt for it in the first place)
Why AI fails at writing good listicles
Why AI is making us mentally obese
Stay tuned!
Besides the above, what more would you like to know? Let us know your thoughts and opinions in the comments below.
December 4 – ChatGPT has more than 1 million users
After one month – 57 million users
After two months – 100 million users
For reference, TikTok took 9 months to reach that milestone, and Instagram took about 2.5 years.
However, the growth of ChatGPT is not unprecedented. People who have been keeping tabs on LLM (Large Language Model) space know that AI writers have existed in good capacity since 2020 – i.e., the year when OpenAI launched GPT-3.
The only difference now is the “accessibility” (and, of course, a wee bit of improvement in quality). ChatGPT has worked to supercharge the market that otherwise relied on hyper-targeting, expensive subscriptions, and a handful of use cases.
We have been analyzing this market since before the venerable (yet arguable) entry of ChatGPT (and now Google’s Bard). And we’ve always been curious about the following:
Are AI writers any good for B2B content?
Can they replicate (or perhaps emulate to a certain degree) the technical nuance of humans?
Are they fit for consumer-facing content in a space rife with a knowledgeable audience?
And if they are any good – what does it mean for businesses and writers out there?
Our Research Was an Eye-Opener
We officially started our assessment of the AI writing tools and their viability for the B2B tech space in October 2022.
We purchased the subscription to five GPT-3 AI writing tools. The choice of the tools was based on a variety of factors, including their reviews and ratings on G2 and Capterra, the support for use cases that complemented long-form writing, the perceived output quality as seen on YouTube, and more. We took the “cost” into account as well, but there was a substantial variation in what these tools were charging, so there was initially no bar set.
Through the course of four months (from October 2022 to January 2023), we used these tools to write 50 articles for us across the most pertinent technology themes relevant to our business – Cloud, Data Science, Software Testing, Digital Transformation, Application Development, Cybersecurity, Industry 4.0, CX Transformation, etc.
Our aim was to understand how the AI writers performed when it came to:
Accommodating the linguistic adeptness for different types of articles: How-to Guides, Long-form Listicles, Thought Leadership Write-ups, What-is Guides, the Why(s)
Accommodating the technical adeptness for Moderately technical (or non-technical), Technical, and Highly Technical write-ups.
For ease of comprehending how the AI performed in terms of linguistic and technical adeptness, we used “time taken” and “usage of tool” as two defining variables and fixed the content length at 600 words.
What Did We Learn?
Massive Human Involvement
Clarity, correctness, coherence, and relevance are critical to the success of both theoretical and analytical B2B write-ups. When it came to AI, the articulation was good; however, it required:
A concrete outline to start off
Rigorous fact-checking
Non-stop interventions for setting the direction
Meticulous morphological interventions to sustain the brand’s tone of voice
Exceptional domain knowledge on the part of the human editor
Scalability Potential
Of course, AI was a clear winner here. The speed of responses and the associated ideas complemented the writer’s workflow and warded off writer’s block – provided the writer was equipped with a comprehensive content brief.
Huge Category (& Thematic) Variation
As outlined above, our focus was to write articles around Cloud, Data Science, Software Testing, Digital Transformation, Application Development, Cybersecurity, Industry 4.0, CX Transformation, etc. Of course, that entailed testing AI on technical articles and against different content types. The results, as seen above, were expected (yet intriguing). For example, AI output was dubious and very surface level for listicles – something that we didn’t see coming, considering that listicles tend to have a more concrete structure in place as opposed to other categories.
Why was this the case? There are a few reasons that we discerned. We’ll explore them in detail in the forthcoming weeks. Stay tuned!
Enter AI DETECTION
In December 2022, ChatGPT rose to utter prominence, inviting AI Detection to the fore as well – which has grown substantially in the academic space, if you will, and understandably so.
But we were again curious to understand the implications in the B2B space. Can such detectors help identify what’s now called “AI Plagiarism.” Heard of CNET’s AI Journalist committing plagiarism? Well, that’s a story in itself.
For the purpose of clarity, we ran human-generated, GPT-3-generated, and ChatGPT-generated content through an AI detector – named GPTZero and later through the newly released OpenAI AI Classifier.
Here’s how the results compared on GPTZero for the definition of “Platform Engineering”
Human-written definition
Here’s a definition we wrote for one of our clients:
“Platform engineering refers to the development of Internal Developer Platforms (IDPs) or engineering platforms that developers, data scientists, or end users can use to speed up application delivery. Essentially, IDPs act as a self-service operational layer between users and the backend services powering the platform. The idea is to modernize application development and realize intended business outcomes at speed.”
ChatGPT-written definition
GPT-3-Powered AI tool’s definition
“Platform engineering is the process of designing, delivering and running digital platforms in order to create economic, social and environmental value. Platform engineers are cross-functional, with a focus on the enterprise and business value of the platforms they’re designing and managing.”
Well, there you have it! How would the AI Content Detection space serve the B2B landscape and SERP? That remains to be seen. More on that in the forthcoming weeks.
Interesting Times Ahead
Of course, as everyone has been proudly (but fearfully) touting, this is just the start. With Google’s Bard releasing in a matter of days, this space will explode even further.
However, two things haven’t changed as much as many had envisioned:
Consumer-facing B2B content isn’t legibly supported by AI unless a good writer or editor (with domain expertise) is controlling the narrative.
AI writers are great companions, but thinking of them as a “replacement” for human expertise is far-fetched.
Surely, technology keeps getting better. But that means that human-generated authoritative, journalistic-style, opinionated, brand-centric, and consumer-centric content has even more relevance. Food for thought?
More to come in this series around:
Things that AI is good at
Things that AI’s Bad at
How can you detect AI content?
Why AI fails at writing thought leadership content? (and why you shouldn’t opt for it in the first place)
Why AI fails at writing good listicles
Why AI is making us mentally obese
Stay tuned!
Besides the above, what more would you like to know? Let us know your thoughts and opinions in the comments below.