OpenAI recently introduced Sora, a new generative AI system that can create short videos based on text prompts. Although Sora is not yet accessible to the public, the high quality of the sample outputs has generated both excitement and concern.
The sample videos released by OpenAI showcase Sora’s capabilities, including prompts such as “photorealistic closeup video of two pirate ships battling each other as they sail inside a cup of coffee” and “historical footage of California during the gold rush.” These videos, created directly by Sora without modification, exhibit impressive visuals, textures, scene dynamics, camera movements, and consistency.
OpenAI CEO Sam Altman also shared videos on X (formerly Twitter) that were generated by Sora in response to user prompts, demonstrating its capabilities.
Sora operates using a “diffusion transformer model,” which combines features of text and image generating tools. Transformers are neural networks commonly used in language models, while diffusion models form the foundation of many AI image generators. Sora utilizes tokens representing small patches of space and time to establish coherence and consistency between frames in a video.
While Sora is not the first text-to-video model, it surpasses previous models in several aspects. It can generate videos with resolutions up to 1920 × 1080 pixels and various aspect ratios, whereas other models are limited to lower resolutions. Sora can also create longer videos and compose multiple shots, distinguishing it from its predecessors.
Both Sora and other models produce broadly realistic videos but may exhibit hallucinations. However, Sora’s videos appear more dynamic with increased interactions between elements. Nonetheless, some inconsistencies become apparent upon closer inspection.
Sora’s potential applications are promising. As video production currently relies on filming or special effects, which can be costly and time-consuming, Sora could serve as a cost-effective prototyping software for visualizing ideas. It may also find applications in entertainment, advertising, and education.
OpenAI’s technical paper on Sora suggests that larger versions of video generators like Sora could serve as capable simulators of the physical and digital world, enabling scientific experiments in various fields. However, experts debate whether achieving this level of simulation is feasible for systems like Sora.
The main concerns surrounding tools like Sora revolve around their societal and ethical impact. The ability to generate realistic videos based on text descriptions could exacerbate disinformation issues, potentially endanger public health measures, influence elections, or burden the justice system with fake evidence. Deepfakes created using video generators may also pose direct threats to individuals, particularly in the form of pornographic content.
Furthermore, questions of copyright and intellectual property arise as generative AI tools require substantial amounts of data for training. OpenAI has not disclosed the source of Sora’s training data, which has been criticized in the past. Authors have even sued OpenAI over potential misuse of their materials.
While these concerns are valid, they are unlikely to impede the development of video-generating technology. OpenAI claims to be taking safety precautions before releasing Sora to the public, including collaborating with experts in misinformation, hateful content, and bias, as well as developing tools to detect misleading content.