Recently I was sort of out on a limb, saying – contrary to the on-trend secular philosophy – that we all are on this face of the earth for a purpose, whether we know it or not.
For the second week in a row, I am climbing that tree again and am going to express an opinion that is different from most people's. The person I am going to offer a rejoinder to is no less than the managing editor of this newspaper, Batuhan Takış, and his recent article, “ChatGPT and I wrote this article.” (If we were on a social media platform, I’d be ornamenting this sentence with lots of smiling faces.)
No, in fact, there is nothing wrong with Mr. Takış’s article; quite the contrary you learn a lot from it. I do not want to put the wrong ideas into the heads of our readers who are going to present a thesis, or a term project report in their classes these days, but the article teaches you how to use the ChatGPT website to write a perfectly passable piece of writing by asking “the same questions several times” and diving its paragraphs and asking again; the “artificial intelligence” (AI) behind the “Language Model for Dialogue” improved its styling and produced a 307-word piece on political correctness, cancel culture and new media in no time.
Mr. Takış does not express his final verdict on the success or failure of AI in writing articles; he simply presents the final product that was even titled by ChatGPT: “Navigating the complexities of political correctness and cancel culture in the age of social media and the internet.” Neither shall I. You may find this short piece perfectly acceptable as an introductory writing on the subject, or it might sound like too much of a concatenated list of concept definitions.
The idea I am going to express is that there is no such thing as artificial intelligence, but there are very intelligent programmers who could be unpopular to some people. I am not a Luddite who opposes new technologies and new ways of working; quite the opposite, I am very fond of new technologies and of the people who keep inventing new ways to utilize them. I have supported the pioneering expansion of computer education since the very day “personal computers” became our new tools for office automation. I will buy the first autonomous vehicle that TOGG, the local and national car maker in Türkiye, is going to offer! I am (well, was) an avid programmer myself, having personally contributed thousands of lines of code to the World Wide Web Consortium (W3C) microblogging codebase, which later turned out to be the basis of what we call today “social platforms,” such as Twitter, Instagram, Facebook, Pinterest and so on.
So, when I say there are extremely intelligent programmers out there who could “train a language model” for dialogue using billions of “tokens,” I know what I am talking about. You may disagree, of course, but let me explain why I am willing to challenge the new universal truth of humanity, saying there is no artificial intelligence but intelligent programmers and their intelligent algorithms.
By the way, do you know that the man who enlivened the term “algorithm” is a Turkish scientist: Muhammad ibn Musa al-Khwarizmi? He was the father of algebra, and some sources claim that he was a Persian polymath.
Then, the first question: What is a language model, and how do you generate a model? A language model is a probability distribution over sequences of words. The words “I am having a cup of... ” could be followed by “tea” if you are talking about Turks or “coffee” if the other words in that “token” indicate that you are an American.
There are more than five useful models to generate a probability that could make the order of words acceptable for a regular reader of that language. Language models generate probabilities by training on a language resource consisting of a large and structured set of texts. Nowadays, modern fast supercomputers can do statistical analysis and hypothesis testing, checking occurrences or validating linguistic rules within a specific language territory.
Fifteen years ago, a couple of young computer scientists visited the U.S. communication agency I used to work for and asked us to provide them with the original and translated texts we produce. The agency used to output at least 10,000 lines of daily news stories and they were translated into approximately 50 languages every day. Those people belonged to what we now know as Google Translate. Not only us, but the BBC, Deutsche Welle, Radio France, Radiotelevision Espanola, Russian TASS and Sputnik agencies, Turkish Radio and Television, and several other international broadcasters provided Google with their texts. Add all the translation companies, encyclopedia developers and famous book publishers to them, and imagine the enormity of the textual data accumulated by Google. Not only Google, of course, add Microsoft and thousands of open-source developers as well.
Google and all others used the data to develop their own translation algorithms, and they turned them to the open-source model developers who work on their “generative pre-trained transformers” to create language models that produce human-like text.
ChatGPT, the “model” our managing editor Batuhan Takış used as the co-writer of his recent article, is based on a language model released in 2020 that is said to use “deep learning”: You give an initial text as a prompt, it produces text that continues the prompt. Technically speaking, it is a decoder-only transformer network with a 2,048-token-long context that uses an unprecedented size of 175 billion parameters, requiring 800GB to store.
In computer science, lexical analysis (lexing or tokenization) is the process of converting a sequence of characters (such as in a computer program or web page) into a sequence of lexical tokens (strings with an assigned and thus identified meaning). To do a lexical analysis, you scan a term, later you combine it with what is called a parser, which together analyzes the syntax of languages, web pages, and so forth. That is what your ordinary web search engine is doing when you enter the term and hit the return key on your keyboard when searching for something. The longer the “token” you enter, the longer it takes to search but the more accurate the results.
There are well-known datasets available for model generators and algorithm developers today. If you are using ChatGPT, you may utilize Common Crawl with 410 billion tokens, WebText2 with 19 billion tokens, Books1 with 12 billion tokens, Books2's 55 billion tokens and Wikipedia with 3 billion tokens. (And you thought Wikipedia was the best!)
The larger the token dataset and faster the computers you have, the more human-like the text you return to the prompt your user entered.
ChatGPT’s answers follow-up questions (remember, Mr. Takış said he entered new questions, he asked the same question again and again), despite occasional mistakes, usually returns more acceptable sentences because it is searching with more tokens. They call it “deep learning,” but there is no learning process here: It is only more Googling larger datasets.
The ChatGPT is not “writing” anything either! It is simply putting already written things together. At school, we have a utility website to check student homework if there is significant plagiarism in them. When I ran Mr. Takış’s ChatGPT article in one of them, it found no significant problem with it. But when you search each and every sentence it has, you see that all the text has been indexed previously by Google because ChatGPT used the same indices Google uses.
There are several other models I tested with Mr. Takış’s initial input; InstructGPT, for instance, produced a similar structure. Since I couldn’t repeat his follow-up questions, I was not able to produce exact text, but I guess it is possible if you give the same prompts.
As there are no unsaid things under the sun, there is nothing new if searched in the same dataset!
This brings us to the danger my dear managing editor warns about: the death of the human author. Yes, the author has been dead since 1967 when the French literary critic and theorist Roland Barthes declared in his famous essay “La mort de l'auteur.” But Barthes and Takış talk about different demises here. Barthes said unlike traditional literary criticism we cannot rely on the intentions of an author to definitively explain the "ultimate meaning" of a text; Takış fears that soon computers are going to write up the text, not hale and hearty individuals.
There are ways to find out if a text is produced by a mere machine or your student as the Russian State University for the Humanities (RGGU) has recently learned. A student successfully presented a diploma thesis written by ChatGPT.
If the graduate student “JustA” (@biblikz) did not take to Twitter to share how he swindled his school using ChatGPT to write his diploma thesis on management, probably nobody would have noticed. As you see in Takış’s example, the program, AI or not AI, can produce a coherent text.
The Russian university has now restricted access to ChatGPT and is asking the Russian Ministry of Education to ban it altogether in the Russian Federation.
Noam Chomsky, an American linguist and cognitive scientist, sometimes called “the father of modern linguistics,” expressed his skepticism about GPT-3's scientific value. He said in a television program: “It's not a language model. It works just as well for impossible languages as for actual languages.” He thinks it's useful for some purpose, but he says, “it tells us nothing about language or cognition generally.”
The head of Facebook’s AI Lab, Jerome Pesenti, said GPT-3 is “unsafe,” pointing to the sexist, racist and other biased and negative language generated by the system when it was asked to discuss Jews, women, black people and the Holocaust.
A last note to my dear managing editor: Keep the paychecks ready, you are not going to shake your writers yet!
