Delphi and AI

Recently, I led a workshop in Slovenia where we explored the current state of AI in relation to Delphi programming. (A note to participants: the slides are finally online—apologies for the delay!) The initial results were, let’s say, interesting enough to warrant further study.

Now, let’s see how today’s "state-of-the-art" AIs perform with Delphi programming!

The contenders are:

• OpenAI with the o1-mini model (my testing suggests that it gives better results than chatgpt-4o)
• Ollama with codellama:13b model (the largest model I can run on my NVidia 4090)
• Gemini with gemini-pro (currently pointing to gemini-1.5-pro)
• Claude with claude-3-5-sonnet-latest (at this moment this resolves to claude-3-5-sonnet-20241022)

As I care for the privacy of the code I send to these tools, I'm using a paid version of Gemini.

These four engines I'll be using directly from the RAD Studio, most of the time through the "AI Chat" panel. 

When the privacy wouldn't be a concern, I'll also be using:

• CoPilot via Microsoft Edge (I use it a lot and I like it for general-purpose questions with googlable answers)

All chat logs will be published on GitHub. 

For an introduction, let's start with a simple question:

"Which is the highest Delphi version that you know about?"

Let's see what LLMs have to say (full log) ...

[OpenAI]

As of my knowledge cutoff in October 2023, the latest version of Delphi is **Delphi 11.4 Sydney**. This version continues to build on the features introduced in previous releases, offering enhancements in areas such as:

- **High-Performance Compiler**: Improved compiler optimizations for faster build times and more efficient executables. - **Enhanced VCL and FMX**: Updated visual component libraries for both VCL (for Windows) and FireMonkey (cross-platform) to support modern UI design and high-DPI displays. - **Cross-Platform Development**: Enhanced support for developing applications across multiple platforms, including Windows, macOS, iOS, and Android. - **RAD Server Integration**: Improved integration with RAD Server for backend services and cloud deployments. - **IDE Enhancements**: More robust debugging tools, improved code navigation, and better performance within the Integrated Development Environment (IDE). - **Language Enhancements**: Continued evolution of the Object Pascal language with new features and improvements for better developer productivity. For the most accurate and up-to-date information, it's always a good idea to visit the [official Embarcadero Delphi product page](https://www.embarcadero.com/products/delphi) or check their latest release notes.

[Ollama]

The latest Delphi version is Delphi 10.4, which was released in September 2020. It is a major update to the Delphi IDE and includes many new features and improvements, such as a new code editor with syntax highlighting and code completion, improved support for modern hardware and software technologies, and a new user interface with a more intuitive layout.

[Gemini]

Delphi 11.2

[Claude]

As of my knowledge cutoff in October 2023, the latest version of Delphi is **RAD Studio 11.4**, which is part of the Delphi 11 "Alexandria" series. RAD Studio 11.4 includes a variety of updates and improvements across the Delphi and C++Builder environments, enhancing performance, adding new components, and improving support for modern operating systems and platforms. Key features of RAD Studio 11.4 include: - **Enhanced FireMonkey (FMX) Framework:** Improved cross-platform capabilities for building applications on Windows, macOS, iOS, and Android. - **VCL Updates:** Continued enhancements to the Visual Component Library (VCL) for Windows development. - **Language Enhancements:** Updates to the Delphi language, including improvements in the compiler and new language features. - **Platform Support:** Better integration and support for the latest versions of operating systems and development platforms. - **Tooling Improvements:** Enhancements to the IDE, debugging tools, and performance profiling tools to streamline the development process. Embarcadero, the company behind Delphi, regularly releases updates and patches, so it's a good idea to check their [official website](https://www.embarcadero.com/products/rad-studio) or contact their support for the most current information on Delphi versions and features. If there have been any releases beyond RAD Studio 11.4 after October 2023, I would not be aware of them. I recommend visiting Embarcadero’s official website or their community forums for the latest updates.

[CoPilot]

The latest version of Delphi is Delphi 12.2, released in September 2024. This version includes various enhancements and updates, such as improved IDE features, better support for modern platforms, and numerous bug fixes. Are you planning to upgrade to the latest version, or do you have any specific questions about it?

Analysis

We got some very different answers, but we can still pull some information from them:

• [OpenAI] and [Claude] like to talk. A lot. [Gemini] is extremely brief. [Ollama] and [CoPilot] are "just right".
• [Ollama] has the oldest "cutoff date" for information. [CoPilot] works with the current state of the web. Other have data cutoff somewhere in 2023 (even though the Claude model is supposed to be generated on 2024-10-22).
• Numbers are suspicious - both [OpenAI] and [Claude] mention Delphi 11.4 which never existed.

I had asked the same question again (and again and sometimes again) to verify how stable the answers are. Full answers are in the log, here are the (dis)honorable mentions:

• [OpenAI] Mostly generates the same answer with variations such as "Delphi 11.6 Sydney" :)
• [Ollama] The answer is also somehow randomized and I got answers such as "I'm a large language model, I am not aware of any specific version of Delphi.", "The highest Delphi version I am aware of is Delphi 10.3, which was released in December 2018.", and "I'm just an AI, I don't have personal experiences or knowledge of specific Delphi versions. However, I can tell you that the latest version of Delphi is currently Delphi 10.3, which was released in 2019." 
• [Gemini] The numbers vary: "Delphi 11 Alexandria (version 29.0)", "Delphi 11.3 Alexandria"
• [Claude] Sometimes knows about 11.4, sometimes it is not sure: "The latest major version of Delphi that I'm confidently aware of is Delphi 11 Alexandria, which was released by Embarcadero Technologies in 2021. However, since my knowledge may not be completely up to date, there might be newer versions available. "
• [CoPilot] Is very sure about version being 12.2, less about the actual year: "The latest version of Delphi is Delphi 12.2, which was released in September 20241." :)

To recap, never trust the numbers generated by "chat" AIs! You can as easily get a completely wrong answer as a correct one!

Just for the sake of completenes, the correct answers are:

• The latest Delphi at this moment is indeed version 12.2. It was released in September 2024
• The highest Delphi 11 Alexandria version was 11.3 and was released in February 2023
• Delphi 10.3 was released in 2018 (and updated in 2019)
• Delphi 10.4 was released in 2020