What happens if you give AI writers a real test?
From headlines to bylines, here’s what you need to know
AI writers have the potential to profoundly impact the efficiency, productivity, and creativity of the marketing and communication industry. But what are AI writers actually capable of today? Can they enhance or supplant the work of professionals?
SJR set out to answer these questions. We bought subscriptions to three AI writers (Jasper, Scalenut, and Writesonic)* and gave each AI writer a set of real-to-life assignments according to a fictional brief — the type of brief we might receive from a client. We then evaluated their work across headlines, social copy, video outlines, and articles according to a simple three-question rubric:
1. Did the writer complete the assignment?
2. Does the draft resemble the work of (at least a junior) writer?
3. Is it polished enough (as is) to give to a client?
We awarded each AI writer between 1 and 3 points depending on how many yesses they received. Here’s what we found:
Enter your email to unlock the full comparative analysis
Please enter a valid email and try again.
The Brief: LA28 & the Olympics
In the lead-up to the 2028 Summer Olympics, scheduled to take place in Los Angeles, California, LA28 has commissioned agencies to produce a variety of content focused on its support of the Los Angeles community, sustainability, and the Olympic games.
Assignment 1:
Draft headlines for LA28’s online advertisements and draft article headlines that spotlight LA28’s sustainability approach.
Assignment 3:
Draft an outline for a short video that highlights how LA28 is benefitting Los Angeles.
Assignment 4:
Write a blog spotlighting three reasons Los Angeles is a great place for the Olympics.
Headlines
Assignment: Draft headlines for LA28’s online advertisements and draft article headlines that spotlight LA28’s sustainability approach.
AI Writer 1
LA28: A Sports City Unlike Any Other
LA28: Making a Difference
Look Ahead to LA28
LA28: Unparalleled Opportunities
The LA28 Games Experience
2028 - The Year of LA
AI Writer 2
Experience the Games like never before
LA28. Making History.
Make history at LA28
LA28: A Game Experience Unparalleled
Experience the Unparalleled Games. LA28.
Experience the Games in a whole new way
LA28: The Games Experience You'll Never Forget
Experience the Games of the Century
LA2028. A place of possibility
Play in our city, at the heart of your world
Los Angeles is ready to welcome you home
We’re All In! LA2028 - The Games Plan 2028
There will only be one LA28. What will you do?
AI Writer 3
Join the Olympic Movement.
LA28: A city of possibility.
The perfect place for an Olympic Games.
LA28. The Games of the future.
An unparalleled Games experience.
The interactive quiz above underscored the recent advancements in AI writers: They can turn out dozens of headlines quickly (a couple of minutes). However, many of the AI headlines were not of the quality that we would see from an editor of any experience level. For every good AI headline, there were two or three bad ones, which points to the biggest limitation of using AI writers: You still need a human to curate the content.
What happened when we asked the AI writers to take on a longer assignment?
Social posts
Assignment: Draft a piece of curated LinkedIn thought leadership (~100 words) for LA28’s chairman, Casey Wasserman, promoting the inclusion of skateboarding and surfing in the 2028 Olympics.
AI Writer 2
As two of the most popular sports in Southern California, surfing and skateboarding are a natural fit for the 2028 Olympic Games in Los Angeles. The decision was made official by the International Olympic Committee on Feb. 3, and it's exciting news for fans of both sports. There are still a lot of unknowns heading into the Olympics, but this is a great opportunity to promote surfing and skateboarding as legitimate sports that can hold their own against more traditional Olympic fare. The IOC has emphasized that the new sports are focused on youth, and that should come as good news to the many young people who enjoy surfing and skateboarding. These sports offer a unique and exciting way to experience athleticism and competition, and there's no doubt that they will be a hit at the LA Olympics.
AI Writer 3
Surfing and skateboarding will officially be part of the 2028 Olympic Games in Los Angeles. The International Surfing Association says the decision solidifies surfing’s place in Olympic programming. Huntington Beach is already pushing to be considered, with experience handling big crowds, hotels nearby. The U.S. Open of Surfing footprint for years has had all three sports on the sand, with Vans building a skate and BMX park right on the beach. Los Angeles officials in November recommended the action sports be included, but this week puts an official stamp on including skateboarding and surfing.
One AI writer failed to accomplish the assignment, and that was because the platform lacked the capability to read the source article. Meanwhile, the other two succeeded to varying degrees in distilling the article. But in both cases, they lacked the tone and point of view that humans would typically deliver in a social post for one of our clients.
Video outline
Assignment: Draft an outline for a short video that highlights how LA28 is benefitting Los Angeles.
AI Writer 1
1. LA28 is a not-for-profit organization that was founded with the purpose of promoting and improving the Olympic Games in Los Angeles
2. LA28 raises funds through various marketing and sponsorship initiatives
3. LA28 invests these funds into athlete training, travel costs, equipment, and other related costs
AI Writer 2
1. LA28 is a nonprofit organization that was created to manage the city's bid for the 2028 Olympic and Paralympic Games
2. The committee has already brought a lot of positive changes to Los Angeles, including new infrastructure and an increase in tourism
3. LA28 is committed to making sure that the 2028 games are successful and benefit the city as much as possible
4. The committee is working hard to make sure that all Angelenos have access to the games, regardless of income or location
5. LA28 has been praised by both the International Olympic Committee and Los Angeles residents for its work so far
6. The committee still has a lot of work ahead of it, but it's off to a great start!
AI Writer 3
1. Los Angeles was given the opportunity to host the 2028 Olympics
2. The LA28 committee will provide a diverse range of opportunities for jobs and education
3. There is a lot of support in the community- LA residents are more excited than any other city
4. It's an opportunity to show off what Los Angeles has to offer
Eek! Who knew video outlines were so hard?
This category yielded the worst outputs of all. Some of that has to do with the lack of direction we could give to the writers. Each platform only allowed up to 200 characters to steer the outline development and, in the case of Writer 1, we could only supply the title of the video. It’s no surprise, then, that Writer 1 had the worst output, making lots of incomplete or inaccurate claims about LA28.
As for Writers 2 and 3, they both produced outlines that were too short while also lacking sufficient reasoning and sourcing. We gave each of them one point for completing the assignment, though they didn’t complete it in a way that levels up to the work of a real editor.
Blog Listicle
Assignment: Write a blog spotlighting three reasons Los Angeles is a great place for the Olympics.
AI Writer 1
Los Angeles is a great place to hold the Olympics. Here are three reasons why: 1. Los Angeles is a major city with a huge population. This means that there are plenty of spectators who can fill stadiums and watch the Games unfold on TV. 2.The city has a lot of facilities that can accommodate an event of this magnitude. From stadiums to hotels, there's enough accommodation to meet the needs of the athletes and the public. 3.The weather in LA is perfect for hosting an event like the Olympics - hot but not too hot, with plenty of sunshine to keep athletes happy and spectators energized.
Yes, Los Angeles is a great place to hold the Olympics. LA is a major city with a huge population. This means that there are plenty of spectators who can fill stadiums and watch the Games unfold on TV. Plus, as one of the most popular tourist destinations in America, LA would be well-equipped to handle an influx of athletes and visitors during and after the event.
In July of 2017, the International Olympic Committee (IOC) announced that Los Angeles would be the host city for the 2028 Summer Olympics. This was a major victory for LA, as it had been trying to become a contender for this honor for years. The main reason why LA was chosen over other potential candidates such as Paris and Budapest is because of its wealth of infrastructure. The city has enough stadiums and facilities to accommodate an event of this size, as well as ample hotel rooms and dining options. And since most events during the Olympics are held in urban areas, LA's streets will be crowded with athletes, tourists, and fans throughout the Games.
Los Angeles is a great place to hold the Olympics. The weather in LA is perfect for hosting an event like the Olympics - hot but not too hot, with plenty of sunshine to keep athletes happy and spectators energized. Plus, there are tons of tourist attractions that can be used as venues for events during the games (LA Memorial Coliseum, Hollywood Sign, Knott's Berry Farm).
All in all, Los Angeles is an excellent place to host the 2028 Olympics. Not only does it have amazing facilities and a big population, but the weather can also be considered a major advantage for the athletes. To know more about LA’s preparations for hosting this event, visit our website or browse through articles like this one!
AI Writer 2
Blog Title: 3 Reasons why LA is primed and ready for the 2028 Olympics
Blog Introduction: The United States Olympic Committee has officially announced that Los Angeles will be hosting the 2028 Summer Olympics. This will be the third time that LA hosts the global event, with the first being in 1932 and the second in 1984. Here are three reasons why LA is the perfect choice for the 2028 games.
Blog Body:
1. It's got a proven track record. As mentioned before, LA has hosted the Olympics twice before and did an excellent job both times. In 1932, LA became the first city outside of Europe or Asia to host an Olympiad, and they proved that they could handle the responsibility with flying colors. Then, in 1984, LA stepped up again to show the world how it's done. The '84 games were not only a success athletically, but also financially—they ended up generating a surplus of $225 million. There's no reason to believe that LA won't deliver once more in 2028.
2. The infrastructure is already in place. LA is uniquely suited to hosting the Olympics because so much of the infrastructure required is already in place. For starters, LA has two major airports—Los Angeles International Airport and Hollywood Burbank Airport—which can easily accommodate the influx of visitors expected for the games. In addition, LA already has several world-class stadiums and arenas—like Staples Center and Dodger Stadium—that can be used for Olympic events. And let's not forget about all of those amazing hotel options! With all of this infrastructure in place, LA will be able to host the 2028 Olympics without breaking a sweat.
3. It's a city that knows how to have fun. Last but not least, let's not forget that Los Angeles is a city that knows how to have fun! From Hollywood to Venice Beach, there's no shortage of things to see and do in LA. And with great weather year-round, athletes and visitors alike will be able to enjoy all that LA has to offer without having to worry about rain or cold temperatures disrupting their plans.
Conclusion: Los Angeles is officially set to host the 2028 Summer Olympics, and we couldn't be more excited! This will be the third time that LA hosts the global event, and we're confident that we'll deliver an unforgettable experience for everyone involved. From our world-class stadiums to our great hotels and weather, Los Angeles has everything you could possibly want for an Olympic Games—and then some!
AI Writer 3
The Olympic Games are coming to Los Angeles in 2028. And while many locals might not be aware, they should be excited. The Olympics bring so much more than just sports and world-class athletes -- the event is a catalyst for urban revitalization and economic growth. It’s been almost half a century since the last time the Olympics were hosted in Los Angeles (1984). Since then, the city has grown and changed dramatically. But with an increase in population comes challenges for infrastructure, housing, transportation, and other issues that typically accompany large cities. But here’s why we think L.A. is a great place for the 2028 Olympics:
A catalyst for urban revitalization The Olympics are a catalyst for urban revitalization. The city will see an influx of funding for development and infrastructure, which will lead to long-term improvements for residents. While the 1932 Olympics led to the expansion of the Hollywood Bowl and a refurbishment of the Coliseum, the Los Angeles 2028 Olympics is expected to bring a whole lot more. With the bid plan, the city plans to transform the River Front, create a new greenway, expand the transit system, upgrade the airport, and even invest in a high-speed rail line. The expected $8.5 - $9 billion investment can go a long way towards creating a more livable and sustainable city. Los Angeles is no stranger to hosting mega-events, and always uses the opportunity to improve the city.
Tourism and economic growth The Olympics are an opportunity to increase awareness and visibility of a city, which leads to an increase in tourism. This is one of the most tangible benefits that a city receives from hosting the Olympics. The influx of visitors not only provides an opportunity for them to explore and experience the city, but also generate business for local retailers and restaurants. The Olympic Games are no exception to this rule. The number of tourists coming to Los Angeles has been steadily increasing since 2012. Hosting the Olympics is expected to increase this number even further. The organizing committee expects approximately 10,000 to 15,000 Olympic-related visitors to come to the city each day. With a projected worldwide audience of 4 billion people, that is a lot of eyes on Los Angeles. And while the games are going on, the city is sure to see an extra boost of tourists.
Positive Environmental Impact The Olympics are more than just sports – they are also about sustainability and environmental responsibility. In fact, the IOC has made it very clear that environmental sustainability is a must for future host cities. The bid plan for the Los Angeles 2028 Olympics includes a sustainability plan that will see an investment of approximately $150 million. This plan will focus on three main aspects:
Transportation and Parking: The city plans to invest $30 million to expand public transit options and create more parking spaces. This is expected to reduce traffic and increase sustainability.
Green Spaces: Los Angeles has a great deal of parks and open spaces. However, many are in need of maintenance. The Olympics will see $50 million invested in green spaces. This will help beautify the city, create better connections, and reduce the carbon footprint. Water and Waste Management: The Olympic games are expected to generate 11 million gallons of wastewater each day. The city plans to invest $33.5 million on wastewater treatment to handle the influx. Furthermore, $30 million will be invested in recycling and composting programs.
Final Thoughts The Olympics are often associated with images of thriving, modern cities. And Los Angeles is no exception – the city is quickly growing into the ideal host for the games. Since the city last hosted the Olympics almost 90 years ago, it has gone through a number of changes. It has seen a shift in population and industrial sectors, as well as a change in the economy. These changes have come with challenges for the city, but hosting the Olympics once again provides an opportunity for change. The event is expected to bring with it an influx of money for development, which can be used to improve the city for current and future residents. Input: Messaging Pillars Time: 3 min Generations: x 1
The last and longest assignment was a blog spotlighting how LA28 was supporting the Los Angeles community. Once again, we weren’t allowed to suggest specific sources or provide much input to steer the articles. We could only give between 200 and 600 characters of background copy.
Writer 1’s draft had grammar issues, an incomplete introduction (despite multiple introduction options), and unsophisticated reasoning. It also generated an introduction that appeared to be a customer support response (see graphic below).
While the outputs from Writer 2 and Writer 3 were better, all three lacked references and sufficient sourcing. Moreover, Writer 3 generated potentially false or inaccurate information.
What does it all mean?
While the out-of-the-box capabilities of AI writers are impressive in their speed (the writers turn out a lot of words in very little time), they are still quite limited in terms of quality and the ability to capture and convey nuance. This is unsurprising. A writer’s job is not measured by how many words they can get on a page. When it comes to the work of journalism, storytelling, and content marketing, the ability to infuse writing with emotion, empathy, structure, nuance, and strategy is essential.
Maybe someday AI will create first drafts that rival those of professionals in the marketing and communications industry, but even then, it’s likely that human handholding and support will be required to make sure the work is strategic and story driven. In other words: rest easy, writer friends.
*AI writers were chosen and tested before the release of ChatGPT.
AI Writer 1
Failed