Growth is slowing down, ChatGPT has entered a bottleneck period?

Source: "Deep Burning" (ID: shenrancaijing), author: Li Qiuhan, editor: Wei Jia

Original title: "ChatGPT can't move up"

Image credit: Generated by Unbounded AI tools

Are you still using ChatGPT?

ChatGPT, which has set off an AI boom around the world, seems to have entered a bottleneck period.

The first is about the usage rate of ChatGPT, there are unfavorable data. In early June, a survey released by Morgan Stanley showed that only 19% of respondents said they had used ChatGPT before, and only 4% said they relied on ChatGPT. According to the survey, the proportion is surprisingly low.

The survey was conducted in April this year, involving 2,000 people. However, in the face of the world's 7.8 billion population, such a sample size is not large, and it also reduces its reference to a certain extent.

There is also data from a broader base worth noting that the growth of ChatGPT has slowed down significantly.

According to data from the website data analysis tool SimilarWeb, the growth rate of ChatGPT’s visits in the early stage was astonishing. The month-on-month growth rate was 131.6% in January, 62.5% in February, and 55.8% in March. It slowed down significantly in April, with a month-on-month growth rate of 12.6%, and by May, this figure had become 2.8%.

With the popularization of ChatGPT, the reference base becomes larger, and it is normal for the growth rate to slow down. However, according to the current trend, the month-on-month growth rate in June may also be negative.

At the beginning of this year, ChatGPT was like a thunderbolt, allowing the world to see the power of generative AI, and also made the GPT (generative pre-training Transformer model) behind it popular, setting off a wave of large-scale model entrepreneurship. It has refreshed a lot of numbers, the most impressive of which is the fastest growing consumer application in history. Only two months after its launch, ChatGPT's monthly active users have exceeded 100 million.

But it is difficult for even the creator to give a clear answer to its future development. Helen Toller, a member of the OpenAI board of directors, once said, "Even the people who created them don't know what they can and cannot do. I predict that we really understand all the things that GPT-4 can and cannot do. It will take a few years."

The current ceiling of ChatGPT does not mean the ceiling of GPT, but as a product supported by the most powerful large language model at present, the trend of ChatGPT can also become a window to observe the application of GPT. The fantasies about AI are still going on, and nearly half a year has passed. What we are curious about is, how is the usage of ChatGPT? Is it overrated?

01**Is ChatGPT really used by many people? **

Regarding the experience of using ChatGPT, different industries and different people have different answers. Some people use it as a toy and stop logging in after using it once or twice; It's "not good enough".

Xia Nan belongs to the third category. She is engaged in the foreign trade industry and can use ChatGPT to write work emails and let it answer some troubles in life. In order to better use ChatGPT, she sends instructions in English.

Since using ChatGPT in February, her experience has been divided into three stages. At the beginning, she was curious and wanted to throw many questions to ChatGPT to see how it answered and explore it. Since May, she feels that ChatGPT has become "stupid". What she could do before, she can't do now. Now, her evaluation of ChatGPT is "not easy to use".

For example, recently, their company took over the ODM (Original Equipment Manufacturing) business of a cooking robot. She wanted ChatGPT to provide forecast data for this market. After repeated pushes and pulls, ChatGPT still did not give her an answer. As for writing work emails, after training, ChatGPT only gave her an imperative follow-up email, and long-winded texts in the official style were not what she wanted. She hoped that "it can write polite, informative Very clear email".

She feels that the reason why ChatGPT can't do it is that it "doesn't understand the world". Not getting the results she wanted, and she used it less frequently, from five or six times a week to once a week.

Of course, there are many comprehensive influencing factors behind such an experience, which is related to whether the user has asked about the field that ChatGPT is good at, and whether the user has found a suitable communication method with ChatGPT.

Lucy, who is living in Australia, has been using ChatGPT in English since it was launched at the end of last year. Now, she uses ChatGPT every day to sort out her thoughts on academic research and learn languages, which is an indispensable tool in her life. However, the problem of accuracy has always troubled her, and the literature reference needs to be found by herself, "If I question its answer, it will answer along my train of thought."

In addition to the different experience of using ChatGPT, judging from the data, the penetration rate of ChatGPT is not as wide as imagined for the time being.

Screenshot of Morgan Stanley related report

In addition to Morgan Stanley's report, there are some data that can be used as evidence. According to SimilarWeb data, from March to May, the United States and Japan are the countries with the highest traffic share for ChatGPT in the world, ranking first and third respectively. Recently, however, traffic in the US has dropped by 10.28%.

ChatGPT's global traffic distribution source/SimilarWeb data

In the United States, at the end of May, the Pew Research Center released a survey. They conducted a survey among more than 10,000 American adults in mid-March this year. 18% of them had heard a lot about ChatGPT, and 39% of people have heard of it a bit, and 42% have not heard of it at all.

In Japan, according to the latest survey report of MM General Research Institute, a Japanese ICT market research and consulting agency, during the period from May 24th to 31st, 13,814 employees belonging to Japanese and American companies (including 13,412 in Japan and 402 in the United States) According to the results of the online survey conducted, the ChatGPT usage rate of Japanese companies is only 7%, compared with the 51% usage rate of American companies, the difference between the two is as high as 44 percentage points.

In Japanese companies, nearly half of employees (46%) answered "don't know" ChatGPT, and even if they know ChatGPT, the proportion of "not using" is also 42%.

These are all recent reports, with a sample size of around 10,000 people. However, after the popularity of ChatGPT, there are many reports on its use around the world, with different opinions, and some even draw opposite conclusions. The above reports are for reference, but due to differences in regions and groups of people, they may not fully reflect the real situation.

There are also more clear and comprehensive data worthy of reference, which can help us understand the application status of ChatGPT.

According to SimilarWeb, the growth of ChatGPT has slowed down significantly, especially until June. As of June 20, two-thirds of June has passed, and the number of visits is about 38% less than that in May. Roughly, by June 31, if there is no Especially with the new stimulus, the month-on-month traffic in June may drop.

ChatGPT recent traffic change source/SimilarWeb data

At the same time, you can also refer to that, according to SimilarWeb, in May, the bounce rate of ChatGPT was 12.59%, lower than that of Google, Youtube, etc., and on June 24, the bounce rate had risen to 37.37%. The average visit duration also dropped from 8 minutes and 32 seconds to 7 minutes and 48 seconds.

Another data is the change of Bing's market share after accessing the GPT large model.

Source/Statcounter

Bing's market share attracted attention when GPT was first introduced in February and March. According to Statcounter, a website communication traffic monitoring agency, Bing's market share in March 2023 was 2.86%, and in May it was 2.77%. Not only did the proportion not increase, but even There is also a downward trend.

02** What limits ChatGPT? **

The problems in the application of ChatGPT have been talked about for a long time, but the impact of these problems on the popularity of its application may be wider than imagined.

The first is the "stupid" thing.

In early June, the voice of "ChatGPT has become stupid" sparked discussions. However, Logan Kilpatrick, the OpenAI developer promotion ambassador, responded that since the release of GPT-4 on March 14, the ontology of the large model has been static, and there is no large amount of external data polluting the model. At the same time, he also admitted that the large model itself is unstable, so there are inconsistencies in the answers to similar prompt words.

An AI practitioner told Shenran that in May, foreign practitioners shared articles proving that GPT was becoming stupid in the OpenAI forum. Recently, he used GPT-4's API to test it and let it do simple calculation problems. Judging from the accuracy of the results, GPT-4-0314 scored full marks, GPT-4 scored 80 points, and GPT-4-0613 barely scored 50 points. Among them, 0314 and 0613 refer to the snapshots on March 14 and June 13 (referring to the state of the entire system at a certain point in time). This result gave him a feeling that GPT-4 was being weakened.

OpenAI's latest version, GPT-4, is worse than GPT-3.5 in outputting information, according to an expert analysis by regulator NewsGuard. In a report released in March of this year, NewsGuard mentioned that GPT-4 not only answered completely false news narratives when prompted by its researchers, but also answered worse than GPT-3.5.

In the view of the above-mentioned AI industry practitioners, the result of this change is that users need to become more specific and proactive to guide GPT-4 in order to obtain the same answer quality as in the past.

This also affects the threshold of ChatGPT's use again, which deviates from the original intention of ChatGPT.

At the very beginning when ChatGPT exploded, some people in the industry analyzed Shenran. The impact it brought was to put general artificial intelligence in front of every user, and it also lowered the threshold of human-computer interaction to the lowest point.

But for now, the threshold still exists. From the user portrait of ChatGPT, we can also see the popularity of this product. According to SimilarWeb data, users are mainly distributed in the computer electronics and technology industries, among which programming and software development account for the largest proportion. Among other industries, only video game consoles and accessories in the game industry account for a large proportion of practitioners.

In the experience of using ChatGPT, an engineer gave Shenran the most positive feedback, saying that he has been using it all the time, "it can help me solve small program problems."

Although it is said that "it is not AI that eliminates you, it is people who can use AI", if the threshold for ordinary people to use is getting higher and higher, it will also deviate from the original intention of ChatGPT to a certain extent.

There are two other issues that ChatGPT has faced since the very beginning, namely accuracy and privacy protection.

According to the report of the relevant Japanese agency mentioned above, when asked what problems need to be solved in order to continue/expand the use of ChatGPT in the future, 49% of Japanese companies and 45% of American companies answered "the accuracy of the case", followed by It is "personal data and other privacy (34% of Japanese companies, 35% of American companies)", and "understanding of the problem (33% of Japanese companies, 34% of American companies)".

In terms of accuracy, OpenAI CEO Sam Altman also explained that this program will confidently claim that something is true, but in fact it is fabricated, just like a politician who is full of lies. He gave this phenomenon a name - "the hallucination problem".

In short, it is not easy to achieve the accuracy rate. The reason is because it works not by memory but by deductive reasoning. "Large language models rely on Scrabble for reasoning. They cannot be as accurate as databases, and humans cannot guarantee accuracy," Yang Yang, an engineer who focuses on the AI industry, told Shenran.

In terms of privacy, OpenAI has not yet given a clear solution. Xiaohong, who works in Canada, told Shenran that the company specially sent an email notification to let everyone use ChatGPT with caution.

Based on these limitations, the application scenarios of ChatGPT are also limited.

Chen Momo, an investor who pays attention to the industry, told Shenran that it is actually suitable for "productivity-driven" content production rather than "creativity-driven" content production. In the former, it can replace a lot of manpower with accumulated experience.

User Luoluo has been using ChatGPT since April. She opened a membership, which is mainly used to write scripts and copywriting. "As long as I can give it the correct formula, I can basically give me feedback on the script of any thinking, but I just need to change it when I get it." She said that the script produced is relatively basic and cannot be made into a hit, but there is no problem with its logic. "It is possible to support some of the company's daily large-scale video output." Her current ChatGPT usage frequency is basically a week. more than 3 times.

Now Xia Nan has adjusted her strategy and only asked it some procedural questions, such as the process of opening an eBay online store. Although you can also ask Google and Baidu for such questions, "ChatGPT's answer is better." She gave an example. She recently traveled to Germany and asked ChatGPT to arrange a travel plan for her. The answers given are informative, and the transportation arrangements are also very good. clear.

These long-term users of ChatGPT, no matter whether they are satisfied with ChatGPT's experience or not, they all mentioned that ChatGPT is more like an upgraded Google and Baidu, which has brought some help.

03**ChatGPT, the symbolic meaning is higher than the real meaning? **

Recently, OpenAI launched an App Store similar to the LLM version to speed up the construction of the ecosystem, and some functional optimizations have also been exposed. There is also a signal hidden behind this. GPT4 has reached the ceiling for the time being. To speed up the ecological construction, we must first do some experience optimization before GPT5 can’t come out.

As early as April, Sam Altman said that he had not started working on GPT-5, nor did he plan to start immediately. He also said, "The era of large models has come to an end."

According to the official website of OpenAI, the number of GPT model parameters (which can be understood as the language material for feeding the model) is constantly increasing. GPT-1 is 117 million, GPT-2 has 1.5 billion, and GPT-3 has soared to 175 billion. GPT-4, according to a report by foreign media agency Semafor, is about six times larger than GPT-3 and has 10,000 billion parameters.

Previously, Yang Yang also told Shenran that maybe GPT-4 has grown to the end, the corpus is one reason, "there are only excellent resources created in human history", and the limitation of the model itself is also a reason. In his view, GPT-4 is now limited, and it should have the ability to not be fully developed.

Recently, Yann LeCun, the chief artificial intelligence scientist of Meta, the parent company of Facebook, pointed out that the generative artificial intelligence technology behind ChatGPT has entered a dead end and has too many limitations.

Due to the competitive relationship among the giants, it may be difficult to use this statement as an objective reference. But what is certain is that ChatGPT has indeed encountered a bottleneck.

In order to make the large language model have a better application, many people turn their attention to the application in the vertical field.

Qin Kai, a practitioner in the AI industry, made a metaphor for Shenran. When generalized artificial intelligence such as ChatGPT is widely used, its ability is like that of high school students and college students. When combined with vertical scenarios, fine-tuning (in natural language The technology used in the processing (also called fine-tuning) data is accurate enough and fits the scene, and the ability can become a master’s or doctor’s degree, which can solve more specific needs.

Yang Yang also agrees with this view. He mentioned that the current model can only be optimized by about double at most. "Everyone has a basic consensus that GPT-5 will not bring about disruptive evolution." It is impossible to achieve AGI (General Artificial Intelligence) in the short term. intelligence) level.

However, he said that to do specific vertical applications now, firstly, the cost is high, and the training model is still not a small cost for the company; secondly, there are issues of data security and data isolation. The current method is, "In the large model Set up a small model based on the basics", but the problem is that the current underlying technology is still changing, "no one knows when the next model will appear, and when a better model will appear", this intermediate stage makes everyone very confused, "if three It will only appear five years later, so it is not a loss to make vertical products based on large models, and there is a chance to recover funds after the scene is implemented. But if it appears soon, then the plug-in vertical products that everyone is making now are not worth it. How meaningful."

Investor Chen Momo said that this is a "chicken or the egg first" question, and they are still willing to look at related projects and cut into a specific scenario application in the segmented field, because "even if the bottom layer changes in the future, As long as the industry does not change, there will still be precipitation in the understanding of the industry at the application layer."

But the problem they encounter when looking at the project is that it is difficult for someone to tell them exactly how much labor costs the product can save. "Looking at it, it is still necessary to assign a person to the machine." She gave an example, focusing on empowering R&D papers to screen and summarize related vertical products. In actual use, one person still needs to follow the results of the machine to do further verification, development and verification. Research, in fact, it is difficult to say that the efficiency has a particularly good optimization", so now, some investors will tend to wait and see.

When paying attention to the products of AI start-up companies in the vertical field, her feeling is, "We are cautiously optimistic about the opportunities for industrial upgrading brought about by technology. At present, its market significance may be higher than its real significance."

Qin Kai concluded that people have high expectations for ChatGPT, but there are two bottlenecks. First of all, the next-generation large language model has diminishing returns through larger parameter scale and stronger computing power, and people's expectations may not be met soon. Second, current large language models are generalizable and take a long time to solve specific, real problems. At present, generative AI in the vertical field has become a manual task for customized needs and private deployment for specific enterprises. "The underlying model relies on the transformer method and lacks the ability to solve very complex problems. The current application situation is far from the expected level. Far".

The application is still going on, the technology is still developing, and the application and potential of ChatGPT still need to wait and see. Even so, ChatGPT has already improved some people's production efficiency by an order of magnitude. Even if there is a bottleneck at the moment, "ChatGPT is already a great product, and that's enough." Yang Yang said.

*At the request of the interviewees, Xia Nan, Lucy, Yang Yang, Xiaohong, and Luo Luo are pseudonyms in this article.