Generative AI
What is Generative Artificial Intelligence and its emergent capabilities
What is in the News ?
• In the wake of newly released models such as Stable Diffusion and ChatGPT, generative AI has
become a 'hot topic' for technologists, investors, policymakers and for society at large.
• Despite the current downturn and layoffs in the tech sector, generative AI companies continue
to receive huge interest from investors.
• While generative AI has people excited about a new wave of creativity, there are concerns about
the impact of these models on society.
What is Generative AI ?
• As the name suggests, generative AI produces or generates text, images, music, speech, code or video.
• Generative AI is not a new concept, and machine-learning techniques behind generative AI have evolved over the past decade.
• Deep learning and General Adversarial Network (GAN) approaches have typically been used, but the latest approach is transformers.
1. Deep Learning : Deep learning is a subset of machine learning, which is essentially a neural network with three or more layers. These neural networks attempt to simulate the behavior of the human brain—albeit far from matching its ability—allowing it to “learn” from large amounts of data.
2. General Adversarial Network : A generative adversarial network (GAN) is a machine learning (ML) model in which two neural networks compete with each other to become more accurate in their predictions. GANs typically run unsupervised and use a cooperative zero-sum game framework to learn.
3. Transformer : A transformer model is a neural network that learns context and thus meaning by tracking relationships in sequential data like the words in this sentence. If you want to ride the next big wave in AI, grab a transformer.
What is GPT ?
• A Generative Pretrained Transformer (GPT) is a type of large language model (LLM) that uses deep learning to generate human-like text.
• They are called ”Generative" because they can generate new text based on the input they receive.
• ”Pretrained" because they are trained on a large corpus of text data before being fine-tuned for specific tasks.
• ”Transformers" because they use a Transformer based neural network architecture to process input text and generate output text.
Emergent Capabilities of Generative AI
• Text to Image : Text-to-image programs such as Midjourney, DALL-E and Stable Diffusion have the potential to change how art, animation, gaming, movies and architecture, among others, are being rendered.
• Generative AI can be used to compose music and create art
• Create brand logo: E.g. many startups are exploring services like DALL.E2, Bing Image Create, Stable Diffusion, and MidJourney to create their brand logo
• ChatGPT to generate news articles, poetry, and even code.
• AI-assisted drug discovery
• Generative AI can be used to design and control robotic systems
• Automate things e.g. Microsoft-owned GitHub Copilot, which is based on OpenAI’s Codex model, suggests code and assists developers in autocompleting their programming tasks.
Growth Potential
• Despite the current market downturn and layoffs in the technology sector, generative AI companies continue to receive interest from investors.
• Stability AI and Jasper, for example, have recently raised $101 million and $125 million, respectively, and investors like Sequoia think the field of generative AI can generate trillions of dollars in economic value.
• Over 150 start-ups have emerged and are already operating in the space.
• By 2025, Generative AI will account for 10% of all data produced.
• The forecasted Al annual growth rate between 2020 and 2027 is 33.2%.
• By 2030, Al will lead to an estimated $15.7 trillion, or a 26% increase in global GDP.
Problematic Issues with Generative AI
• Governance: Companies such as OpenAI are self-governing the space through limited release strategies, and monitored use of models, however, self-governance leaves chances for manipulation
• Fear of Job losses : Automation of tasks that were previously done by humans, such as writing news articles or composing music.
• Reduced need for human cognition : For example young children who will see AI as their friend to do their homework.
• Fear of Societal Bias being replicated by AI.
• Issues surrounding intellectual property and copyright: The datasets behind generative AI models are generally scraped from the internet without seeking consent from living artists or work still under copyright
• Fear of Misinformation and Mistrust by manipulation of information, creating fake text, speech, images or video
• Fear of Concentration of Power in the hand of a few companies
• Risks for national security using automated troll bots, with advanced capabilities
Way Forward
• Need to make generative AI models more transparent, so that the public can understand how and why the model is making certain decisions
• Use of diverse training data, as well as techniques like fairness constraints or adversarial training to mitigate bias.
• Privacy: Ensuring the privacy of people
• Accountable governance esp. of BigTech companies using a designated “AI ethicist” or “AI ombudsman”.
• Designing a system wherein humans make the final decision and AI can be used as a support system
• Collaboration with civil society and policymakers: To mitigate the impact of Generative AI on - the disruption of labour markets, legitimacy of scraped data, licensing, copyright and potential for biased or otherwise harmful content, misinformation, and so on