RTX AI is a broad term used to collectivity describe NVIDIA’s GeForce RTX technologies and its AI capabilities. This includes hardware such as the GeForce RTX 4060 and software that could utilize its potential like the ChatRTX – a powerful GPT large language model (LLM) connected to your own local content.
In this article, we are going to explore RTX powered AI tools with a little help from the Palit GeForce RTX 4060 Infinity 2. This is a new graphics card, recently announced at Palit’s Computex 2024 showcase.
Table of Contents
What is RTX AI?
As described earlier, RTX AI is basically a combination of NVIDIA’s hardware and software capable of generating AI content locally. With its local generative model approach, NVIDIA’s RTX AI not only aims to give generative power to RTX hardware enabled PCs – it also pushes for data privacy, greater security, and customization to name a few.
RTX AI Hardware
RTX AI on the hardware side started with the introduction of the GeForce RTX 20 series, specifically with the introduction of the Tensor cores. These are AI accelerators used initially for DLSS to aid ray tracing’s performance hit – and the rest is history. RTX GPUs are exceptionally well-suited for LLMs due to their memory speed, memory capacity, and essential support for TensorRT-LLM software.
Graphics Model | AI TOPS |
---|---|
GeForce RTX 4090 | 1321 |
GeForce RTX 4080 SUPER | 836 |
GeForce RTX 4080 | 780 |
GeForce RTX 4070 Ti SUPER | 706 |
GeForce RTX 4070 Ti | 641 |
GeForce RTX 4070 SUPER | 568 |
GeForce RTX 4070 | 466 |
GeForce RTX 4060 Ti | 353 |
GeForce RTX 4060 | 242 |
Now AI TOPS quantifies an AI capable hardware’s processing capabilities by checking how many operations it could make in a second. We are looking at trillions of operations per second (TOPS) here which is a scale made easier for humans to comprehend. Take the GeForce RTX 4090 as an example with its 1321 AI TOPs. That’s basically a quadrillion worth of calculations in a span of a second.
RTX AI Applications
Regarding the software aspect of RTX AI, these primarily consist of GPT and LLM-based applications that NVIDIA has developed or co-developed (as features) with other software vendors. These applications or features are specifically designed to function with the RTX series GPUs in mind. The TensorRT-LLM library played a huge part here, acting as the API between the said hardware and applications.
Listed below are the publicly available RTX AI applications that you may try with compatible NVIDIA GeForce RTX hardware:
- ChatRTX – Get tailored responses from local files with your own personal private chatbot. Search personal notes, files, and photos with text or voice.
- NVIDIA Broadcast – Level up your live streams with AI-powered Noise Removal, Background Replacement, and more.
- NVIDIA DLSS – Maximize performance and quality in your favorite games. DLSS uses AI to create additional frames and improve image quality.
- NVIDIA ACE – Bring AI game characters and digital humans to life with generative AI.
- RTX Remix – Easily capture game assets, automatically enhance materials with generative AI tools, and quickly create stunning RTX remasters with full ray tracing and DLSS.
- NVIDIA Canvas – Turn simple brushstrokes into realistic landscape images for backgrounds, concept exploration, or creative inspiration.
- RTX AI Toolkit – Customize, optimize, and deploy AI models for Windows applications with an end-to-end suite of tools and SDKs.
- RTX Video – AI Super Resolution and HDR in Chrome, Edge, and Firefox browsers plus VLC Media Player turn standard internet video into crystal clear 4K HDR media.
As said earlier, RTX AI software are not necessarily NVIDIA in-house solutions alone. They may also be a part or a feature from software vendor’s application capable of harnessing the power of the RTX platform. These includes the following:
- Enhanced AI Effects – Supercharge your video editing process with AI effects and tools in DaVinci Resolve, Adobe Premiere Pro, Capcut, and more.
- Interactive Design – Real-time viewport rendering, upscaling, ray reconstruction, and more are available in 3D creative apps like Adobe Substance, Adobe Painter, Blender, D5 Render and Unreal Engine. Boost rendering performance with NVIDIA DLSS and OptiX AI technologies.
- Stable Diffusion – Generate images and videos faster on NVIDIA RTX GPUs, accelerated by NVIDIA TensorRT™.
Why get a GeForce RTX AI PC?
According to Steam Hardware & Software Survey, NVIDIA dominates the ranking of graphics card shares from the popular gaming platform. The GTX 1650 was once king, and has since been dethroned by the GeForce RTX 3060, while the majority of remaining chart toppers are comprised of other GeForce RTX graphics card as well. On that note, there are now millions of gamers with RTX AI capable PCs – and that’s on Steam alone.
With that in mind, RTX AI-enabled PCs are not only popular for their gaming performance but also for their versatility. Many users, including myself, also uses RTX hardware for various tasks. I personally use RTX AI for Stable Diffusion, RTX Video to improve old Youtube videos, and just recently, ChatRTX for file management and organization. All these, done locally.
Benefits of Local AI Processing includes:
- Performance – Local processing on RTX GPUs offers high performance with low latency, which is crucial for real-time applications like gaming and live broadcasting.
- Privacy and Security – Local AI processing ensures that sensitive data remains on the user’s device, enhancing privacy and security.
- Independence from Internet Connection – Since AI tasks are processed locally, they are not dependent on an internet connection, ensuring consistent performance regardless of network conditions.
- Customizability and Control – Users and developers have more control over their AI applications and can customize performance settings to optimize for their specific needs.
While versatility may not have been the initial reason to purchase a GeForce RTX graphics card, today it plays a significant role in the decision-making process of any prospective buyer looking into enhancing their experience with AI-driven apps.
Take a look at a few examples of what RTX AI could do:
NVIDIA DLSS 3.5, ACE and RTX Remix for Gaming
NVIDIA DLSS is the poster child of the RTX AI PC initiative. It basically works by using Tensor cores to render high-resolution images using low-resolution images outputted by your GPU. Previously, GPUs were responsible for this task while also calculating physics and rendering complex scenes, so delegating this task has provided GPUs with the needed breathing room to allocate resources more efficiently. That equates to more frames and is especially useful for intensive titles.
GeForce RTX 4060 vs RTX 3060 | |||
---|---|---|---|
Graphics Model | RTX 4060 | RTX 3060 | RTX 3060 Ti |
DLSS | DLSS 3 | DLSS 2 | DLSS 2 |
Shader Cores | Ada Lovelace 15 TFLOPS |
Ampere 13 TFLOPS |
Ampere 16 TFLOPS |
Ray Tracing Cores | 3rd Gen 35 TFLOPS |
2nd Gen 25 TFLOPS |
2nd Gen 32 TFLOPS |
Tensor Cores (AI) | 4th Gen 242 AI TOPS |
3rd Gen 102 AI TOPS |
3rd Gen 130 AI TOPS |
NVENC | 1x 8th Gen with AV1 |
1x 7th Gen | 1x 7th Gen |
VRAM | 8 GB GDDR6 |
12 GB GDDR6 |
8 GB GDDR6 |
A GeForce RTX 4060 for example might be a midrange card in terms of its raw raster performance, but with DLSS enabled, it could rival higher end GPUs. PALIT’s GeForce RTX 4060 Infinity 2 for example, could easily beat the GeForce RTX 2060 and even the GeForce RTX 3060 Ti with DLSS in tow.
Now NVIDIA ACE is yet another RTX technology that I am waiting for to catch up. ACE, in its simplest form is designed to create realistic and interactive virtual avatars for various applications, including gaming, virtual reality, and digital assistants. Once realized, ACE could open a new chapter for gaming, allowing you to interact with NPCs in a more realistic way. There are countless NVIDIA ACE demos on Youtube on how the tech works but due to its infancy, don’t expect it to become available on future titles soon.
If you’re into modding and classic titles, RTX Remix may pique your interest. RTX Remix is a tool developed by NVIDIA that allows modders and developers to enhance and remaster classic PC games with modern ray tracing and AI-driven technologies. This of course includes DLSS and ray tracing that you could inject on games dating as old as the DirectX 8 API.
Support for Creative Applications
RTX AI also has a special place for creatives with support for a huge number of applications within the creative space. This includes support for Adobe’s Creative Cloud Suite, Autodesk products, Blender, Cinema 4D, DaVinci Resolve and a whole lot more.
I personally use RTX acceleration for Adobe Photoshop’s Neural Filters such as Photo Restoration and Super Resolution to easily restore and upscale images with higher quality compared to time consuming traditional methods.
TensorRT-LLM for Development
As for software development, NVIDIA’s TensorRT-LLM serves as the basis for most RTX AI-powered applications. It is an open-source library designed to accelerate and enhance LLM inference, providing immediate support for widely-used community models such as Phi-2, Llama2, Gemma, Mistral, and Code Llama.
What’s cool is that everyone, regardless of skill, may explore TensorRT-LLM-optimized models within the NVIDIA AI Foundation library. This is out of my interest, but I got to say the ability to learn and develop your own software via community-driven open source library is basically what a modding toolkit is for modders.
On that note, many RTX AI applications, including ChatRTX, have actually been developed using TensorRT-LLM.
AI for Everyday
Microsoft’s push for CoPilot+ had everyone scrambling to put NPUs (Neural Processing Units) into their devices. If you have an RTX AI PC, then this should be trivial to you as they support it from day one.
For those who are still living under a rock, CoPilot+ is a self-contained GPT-4o enabled chat-interface in Windows 11 for natural language processing and visual content recognition. This also includes Recall, image generation and the perks of using CoPilot without requiring an internet connection.
ChatRTX is also something I want to talk about here, as it functions like CoPilot+. While not as powerful as the said software in its most basic form, ChatRTX allows you to build your own custom LLM or a custom chatbot to quickly get contextually relevant answers based on your files. This is extremely useful for file organization, or specific tasks related to let’s say, your work.
For instance, when dealing with thousands of images, you can efficiently organize them using prompts. This method is incredibly beneficial for photographers who need to locate a specific photo without spending excessive amounts of time searching for it. Since ChatRTX is based on TensorRT-LLM, there are also downloadable models that you could try to further enhance your experience – all thanks to its developer and consumer friendly community.
Do I need a high-end graphics card for RTX AI?
Absolutely not. RTX AI will run even on an older GeForce RTX 20 series graphics, but to get the most out of it, I suggest getting at least a GeForce RTX 40 series graphics.
If you do not want to break the bank, the Palit GeForce RTX 4060 Infinity 2 is an excellent choice. Any GeForce RTX 4060 will do just fine really with the point being that it is the entry level point for the RTX 40 series along with its feature set. I.e., support for NV1 encoding, DLSS 3.X, and of course, full support for the current RTX AI ecosystem.
As for performance, a GeForce RTX 4060 is at least 10-20% faster compared to the GeForce RTX 3060 at raster performance with better power efficiency. AI performance on the other hand is a different story, even beating the GeForce RTX 3060 Ti. In fact, the GeForce RTX 4060 will easily beat an RX 6950 XT and could even go head-to-head with an RX 7800 XT at Stable Diffusion – a GPU almost twice its price.
*Data based on Tom’s Hardware’s Stable Diffusion Benchmarks, December 2023.
Final Thoughts
Computex 2024 has shown that AI is here to stay. I’ve seen a lot, even things that I never thought was possible for AI to achieve in a span of seconds. That includes telemetry data from AI powered formula-inspired race cars down to simple everyday chatbots that is almost human-like. What do they have in common? Most of them are running on RTX AI PCs.
Considering these factors, it’s reasonable to conclude that RTX AI PCs are a solid choice for gaining an early advantage in this field. NVIDIA’s commitment to AI is well-established, having invested in the technology early on with the launch of their Volta V100 GPUs in 2017. The competition appears to be lagging, and NVIDIA just won’t settle just yet.