Confluent adds AI tools to secure live data pipelines

Confluent has introduced new tools in Confluent Intelligence and Confluent Cloud to help organisations build and secure real-time artificial intelligence applications. The additions target companies struggling to move AI projects from pilot stages into production.

The updates focus on three recurring obstacles in AI deployment: fragmented data, governance requirements, and the operational burden of managing streaming systems. New features include natural language controls for streaming operations, automatic handling of sensitive personal data, private cloud connectivity, integration with established data engineering workflows, and support for additional AI models.

At the centre of the launch is a managed Model Context Protocol server, Confluent MCP, and a set of Agent Skills. These are designed to let developers build, manage, and debug streaming operations through natural language instead of relying solely on manual commands and separate interfaces.

The aim is to reduce the time developers spend switching between tools to inspect data streams and manage the infrastructure supporting AI systems. Agent Skills are also intended to apply standard workflows and internal practices more consistently.

Another addition is a machine learning function in Flink SQL for detecting and redacting personally identifiable information. It identifies and obscures sensitive data directly in live data streams, without moving information into a separate warehouse or relying on external services.

The feature is likely to interest sectors such as financial services, healthcare, and insurance, where compliance and privacy controls can slow the introduction of AI systems. By handling redaction inside streaming workflows, Confluent is addressing a problem that often leads security teams to restrict the use of operational data in AI pipelines.

Confluent Cloud now also supports Azure Private Link. This gives customers private network paths to external models and external tables, allowing Flink jobs to connect to Azure-hosted services, including Azure OpenAI, Azure SQL, and Cosmos DB, without traversing the public internet.

That private connectivity addresses a common concern for companies building AI systems around large language models and external data services. For many businesses, security rules around public internet exposure remain a barrier to moving beyond test deployments.

Workflow changes

Confluent has also released an open source dbt adapter designed to bring Flink SQL on Confluent Cloud into dbt workflows. The move allows data teams to define, test, and deploy streaming pipelines using the dbt commands and project structures they already use for batch-oriented data work.

The adapter reflects a broader effort to fit streaming technology into established engineering habits rather than requiring teams to adopt separate processes. For organisations already using dbt widely, that could reduce friction in extending existing pipelines to real-time use cases.

Support has also been expanded for TimesFM models, which can be used for anomaly detection, alongside Anthropic and Fireworks AI models. These models can be used in Flink stream processing workflows, giving developers a broader range of options for building AI applications that depend on live data.

The announcement comes as companies continue to invest in AI systems but struggle to put them into live business settings. Confluent cited McKinsey findings showing that data limitations remain a significant obstacle to scaling agentic AI, with many organisations identifying shortcomings in the data layer as a key roadblock.

Sean Falconer, Head of AI at Confluent, linked that challenge directly to how data is handled inside organisations. "Most AI projects fail before they reach a single customer because the data layer breaks down," Falconer said.

"Teams have the models and the mandate, but security risks and fragmented data stop them from shipping. We're fixing that by making the streaming layer the foundation for secure, production-ready AI," he said.

Greg Taylor, Vice President and General Manager for APAC at Confluent, said the problem was particularly visible among businesses trying to move beyond trials and proofs of concept. "In APAC, the race to deploy AI is hitting a hard reality: most projects never make it past the pilot phase because the data layer isn't secure or scalable enough for production. We're changing that. With automated privacy and AI-native tooling, we're giving organisations a clear, secure path to creating real world use cases that actually move the needle for the business," Taylor said.

The update also sits alongside Confluent's broader effort to deepen links with IBM following its acquisition. Its real-time data services are being integrated more closely into IBM's AI and data products, including watsonx.data, as businesses look for ways to combine governed historical data with live operational information.

Confluent's Real-Time Context Engine is now generally available, providing governed context for AI applications from both historical and live data sources.