Deploy an IoT Automation Pipeline with Coreflux MQTT Broker and MongoDB

MQTT brokers are a core component of modern IoT infrastructures and automation environments. They provide a centralized, consistent, and fast data hub that enables interoperability between systems and efficient data exchange. Coreflux is a powerful low-code MQTT broker that extends the classic MQTT broker concept with advanced capabilities for real-time data processing, transformation, and integration with managed databases such as MongoDB, PostgreSQL, MySQL, and OpenSearch.

In this detailed DevOps tutorial, you will deploy a complete IoT automation pipeline by using the Coreflux MQTT broker together with MongoDB on a general cloud infrastructure. This scalable storage and processing setup allows you to collect, transform, and store IoT data efficiently while maintaining reliability and performance suitable for enterprise use cases.

Key Takeaways

Before you start with the step-by-step deployment, these are the main points you will learn:

• Deploy a managed MongoDB database cluster on a cloud platform for scalable IoT data storage.
• Install and run the Coreflux MQTT broker on a cloud virtual machine using a marketplace image or Docker.
• Create secure private networking to connect your MQTT broker and database without exposing them publicly.
• Build real-time data pipelines with Coreflux’s Language of Things (LoT) for low-code IoT automation.
• Automatically transform and store IoT data from MQTT topics into MongoDB collections.
• Verify the complete data flow from simulated sensors through transformation models into database storage.

This tutorial gives you a production-ready foundation for IoT applications that require real-time messaging combined with persistent data storage.

What You Will Build

By the end of this automation guide, you will have deployed:

• A managed MongoDB database cluster for scalable storage
• A cloud virtual machine running the Coreflux MQTT broker
• A Virtual Private Cloud (VPC) network for secure IoT communication
• Real-time data simulation with the LoT Notebook extension
• Low-code data transformation models and database integration routes
• A complete data integration and transformation pipeline for IoT automation

Coreflux and Cloud Platform Integration

Coreflux provides a lightweight MQTT broker and data pipeline tools through the Language of Things programming language, enabling efficient IoT communication in cloud environments.

What Is MQTT?

MQTT, short for Message Queuing Telemetry Transport, is a lightweight publish-subscribe network protocol commonly used in IoT ecosystems. It is designed for constrained devices as well as low-bandwidth, high-latency, or unreliable networks, allowing efficient real-time messaging in environments with limited connectivity.

About Coreflux

Coreflux provides a lightweight MQTT broker for efficient real-time communication between IoT devices and applications. It also includes real-time data transformation features that can be adapted to different use cases. Built for scalability and reliability, Coreflux is designed for environments where low latency and high throughput are essential.

Coreflux delivers the messaging backbone needed to keep data flowing smoothly between devices, whether you are building a small IoT project or implementing a large-scale industrial monitoring system.

With Coreflux in a cloud environment, you get:

• Data Processing: Centralized processing of your data where it is generated and stored, enabling real-time data handling.
• Data Integration: Simple integration with managed database services, helping you create a unified ecosystem for your data requirements.
• Scalability: The ability to support increasing amounts of data and connected devices without sacrificing performance.
• Reliability: Consistent and dependable messaging across all connected devices.

Prerequisites

Before you begin this MQTT broker deployment tutorial, you need:

• A cloud account with billing enabled
• A basic understanding of MQTT protocol concepts and IoT architecture
• Visual Studio Code for the LoT Notebook extension

Create the Network Infrastructure for IoT Automation

Create a VPC Network for Secure MQTT Communication

First, create a Virtual Private Cloud (VPC) to ensure secure communication between your IoT services and MQTT broker without requiring public access.

1. Log in to your cloud provider control panel.
2. Navigate to the networking section and open the VPC area from the sidebar.
3. Click the option to create a new VPC network.

Configure your VPC for IoT automation:

• Name: coreflux-integrations-vpc, or use your preferred VPC name
• Datacenter region: Choose Frankfurt or your preferred region
• IP range: Use the default range or configure it as required
• Description: Add a meaningful description for your MQTT broker and database network

Click the button to create the VPC network.

The VPC provides isolated networking for all IoT resources and ensures secure communication between the Coreflux MQTT broker and managed databases.

Set Up a Managed MongoDB Database for Scalable Storage

Create a Managed MongoDB Cluster

Managed databases provide automated backups, monitoring, and maintenance, making them well suited for production IoT workloads and scalable storage requirements.

Create a managed MongoDB cluster:

1. From your cloud control panel, navigate to the databases section.
2. Click the option to create a database cluster.

Configure your MongoDB cluster for IoT automation:

• Database engine: Select MongoDB
• Version: Choose the latest stable version
• Datacenter region: Select Frankfurt, or the same region as your VPC
• VPC network: Select the coreflux-integrations-vpc you created earlier
• Database cluster name: mongodb-coreflux-test
• Project: Select your target project

Choose your plan based on your IoT requirements:

• For development: A basic plan with 1 GB RAM
• For production: A general-purpose or higher plan for scalable storage

Click the button to create the database cluster.

The creation process for the managed database usually takes between one and five minutes. Once it is complete, you will be redirected to the database overview page, where you can view connection details and perform administrative actions.

Configure Database Access for MQTT Broker Integration

You will be guided through initial setup steps where the connection details are displayed and inbound access rules can be configured. It is recommended to restrict access to your own IP address and to VPC-only communication where possible.

1. Click the getting started option to configure your MongoDB database.
2. Optionally restrict inbound connections.
3. Add your local computer’s IP address for management access.
4. The virtual machine can be allowed automatically through VPC networking.

For connection details, you will usually see two options: public network and VPC network. Public network details can be used for external tools such as MongoDB Compass, while the VPC network details should be used by the Coreflux service to access the database privately.

Note the provided connection details for both public and VPC access, as these are usually different:

• Host: Your database hostname
• User: The default admin user
• Password: The automatically generated secure password
• Database: The authentication database name

Test the Database Connection

You can test the MongoDB connection with MongoDB Compass or the provided connection string by using the public access credentials:

mongodb://username:password@mongodb-host:27017/defaultauthdb?ssl=true

Create an Application Database and User Optional

For improved security and clearer organization, create a dedicated database user and database for your IoT automation application. This can also be done through MongoDB Compass or the command line, but many cloud platforms provide a user-friendly interface for this task.

1. Open the users and databases tab in your managed database cluster.
2. Create a user.

• Username: coreflux-broker-client
• Password: Autogenerated

Create a database:

• Database name: coreflux-broker-data

Deploy the Coreflux MQTT Broker on a Cloud Virtual Machine

Create the Cloud Virtual Machine

1. Navigate to the virtual machine or server section in your cloud control panel.
2. Click the option to create a new virtual machine.

Configure your virtual machine for the MQTT broker deployment:

• Choose region: Frankfurt, or the same region as your managed database
• VPC network: Select coreflux-integrations-vpc
• Choose an image: Go to the marketplace image section
• Search: Search for “Coreflux” and select the Coreflux marketplace image

Choose a size for your IoT workload:

• For development: A basic plan with 2 GB memory
• For production: A basic or general-purpose plan with at least 4 GB memory for scalable performance

Choose an authentication method:

• SSH key: Recommended for stronger security
• Key generation: A key can be created locally with ssh-keygen
• Password: Alternative option

Finalize the details:

• Hostname: coreflux-test-broker
• Project: Select your project
• Tags: Add relevant tags for DevOps organization

Click the button to create the virtual machine.

Open the virtual machine overview page and wait until deployment has finished.

Alternative Installation with Docker on a Docker Image Virtual Machine

Using the same approach as the Coreflux virtual machine, select Docker as the marketplace image.

Once your virtual machine is running, connect to it through SSH with the configured authentication method or use the web console available on the virtual machine overview page:

ssh root@your-vm-ip

Run the Coreflux MQTT broker with Docker:

docker run -d \
  --name coreflux \
  -p 1883:1883 \
  -p 1884:1884 \
  -p 5000:5000 \
  -p 443:443 \
  coreflux/coreflux-mqtt-broker-t:1.6.3

This Docker command:

• Runs the container in detached mode with the -d option
• Names the container coreflux
• Exposes the required ports for MQTT and the web interface
• Uses the current Coreflux image

Verify that the MQTT broker is running:

docker ps

You should see a running container.

Validate the MQTT Broker Deployment with Default Connection Values

You can access the MQTT broker through an MQTT client such as MQTT Explorer to validate broker access, regardless of which deployment method you used.

Configure Firewall Rules for Secure IoT Communication Optional

For production IoT automation deployments, configure firewall rules to restrict access.

1. Navigate to the networking or firewall section.
2. Click the option to create a firewall.

Configure inbound rules for MQTT broker security:

• SSH: Port 22 from your IP address
• MQTT: Port 1883 from your IoT application sources
• MQTT with TLS: Port 1884 for secure MQTT with TLS
• WebSocket: Port 5000 for MQTT over WebSocket
• WebSocket with TLS: Port 443 for MQTT over WebSocket with TLS

Apply the firewall to your virtual machine.

Set Up IoT Data Integration with Coreflux Language of Things

Install the LoT Notebook Extension

The LoT, or Language of Things, Notebook extension for Visual Studio Code provides an integrated low-code development environment for MQTT broker programming and IoT automation.

1. Open Visual Studio Code.
2. Go to Extensions with Ctrl+Shift+X.
3. Search for “LoT Notebooks”.
4. Install the LoT VSCode Notebooks Extension by Coreflux.

Connect to Your MQTT Broker

Configure the connection to your Coreflux MQTT broker with the default credentials when prompted in the top bar or by clicking the MQTT button in the bottom bar on the left.

• User: root
• Password: coreflux

If no errors occur, you will see the MQTT connection status for the broker in the bottom bar on the left.

Create Data in the MQTT Broker with Actions

For this use case, you will build an integration that takes raw data, processes it through a transformation pipeline, and stores it in a database. Since no MQTT devices are connected in this demo, you will use IoT capabilities and create an Action to simulate device data.

In LoT, an Action is executable logic triggered by specific events such as time intervals, topic updates, explicit calls from other actions, or system components. Actions enable dynamic interaction with MQTT topics, internal variables, and payloads, making complex IoT automation workflows possible.

Therefore, you can use an Action that generates data in specific topics at a defined time interval. This data can then be used by the rest of the pipeline configured later.

Generate Simulated IoT Data

Create an Action to generate simulated sensor data with the low-code LoT interface:

DEFINE ACTION RANDOMIZEMachineData
ON EVERY 10 SECONDS DO
    PUBLISH TOPIC "raw_data/machine1" WITH RANDOM BETWEEN 0 AND 10
    PUBLISH TOPIC "raw_data/station2" WITH RANDOM BETWEEN 0 AND 60

The provided Notebook also includes an Action that uses an incremental counter to simulate data as an alternative to the Action shown above.

When you run this Action, it will:

• Deploy automatically to the MQTT broker
• Generate simulated IoT sensor data every 10 seconds
• Publish real-time data to specific MQTT topics
• Show synchronization status in the LoT Notebook interface
• Indicate whether the code in the LoT Notebook differs from the code running in the broker or whether it is missing entirely

Create Data Transformation Models for Real-Time Processing

Define Data Models with Language of Things

Models in Coreflux are used to transform, aggregate, and calculate values from input MQTT topics before publishing the results to new topics. They form the basis for creating the Unified Namespace of your system and can be applied across several data sources.

A Model defines how raw Language of Things data should be structured and transformed, either for a single device or for multiple devices at the same time through the wildcard +. A model also acts as the main data schema used for scalable storage in the managed database.

DEFINE MODEL MachineData WITH TOPIC "Simulator/Machine/+/Data"
    ADD "energy" WITH TOPIC "raw_data/+" AS TRIGGER
    ADD "energy_wh" WITH (energy * 1000)
    ADD "production_status" WITH (IF energy > 5 THEN "active" ELSE "inactive")
    ADD "production_count" WITH (IF production_status EQUALS "active" THEN (production_count + 1) ELSE 0)
    ADD "stoppage" WITH (IF production_status EQUALS "inactive" THEN 1 ELSE 0)
    ADD "maintenance_alert" WITH (IF energy > 50 THEN TRUE ELSE FALSE)
    ADD "timestamp" WITH TIMESTAMP "UTC"

This low-code model:

• Uses the wildcard + to automatically apply to all machines
• Converts energy values to watt-hours, or energy_wh, by multiplying them by 1000
• Determines the production status based on energy thresholds
• Tracks production counts and stoppage events
• Adds timestamps to all real-time data points
• Extracts the machine ID from the topic structure
• Publishes structured data to the Simulator/Machine/+/Data topics by replacing the + with each topic that matches the trigger or source data format

Since two simulated sensors or machines were generated with the Action, the Model structure is automatically applied to both. This produces both a JSON object and the individual topics.

Set Up Database Integration for Scalable Storage

Create a Database Route

Routes define how processed real-time data is sent to external systems such as managed databases. They use the following low-code format:

DEFINE ROUTE mongo_route WITH TYPE MONGODB
    ADD MONGODB_CONFIG
        WITH CONNECTION_STRING "mongodb+srv://:@/?tls=true&authSource=admin&replicaSet="
        WITH DATABASE "admin"

Replace the placeholders with your MongoDB connection details from your managed database platform and run the Route in your IoT Notebook.

Update the Model for Database Storage

Modify your LoT model so that it uses the database route for scalable storage by adding the following lines to the end of the Model:

STORE IN "mongo_route"
    WITH TABLE "MachineProductionData"

In addition, add a parameter with the topic so each entry in your managed database has a unique identifier.

DEFINE MODEL MachineData WITH TOPIC "Simulator/Machine/+/Data"
    ADD "energy" WITH TOPIC "raw_data/+" AS TRIGGER
    ADD "device_name" WITH REPLACE "+" WITH TOPIC POSITION 2 IN "+"
    ADD "energy_wh" WITH (energy * 1000)
    ADD "production_status" WITH (IF energy > 5 THEN "active" ELSE "inactive")
    ADD "production_count" WITH (IF production_status EQUALS "active" THEN (production_count + 1) ELSE 0)
    ADD "stoppage" WITH (IF production_status EQUALS "inactive" THEN 1 ELSE 0)
    ADD "maintenance_alert" WITH (IF energy > 50 THEN TRUE ELSE FALSE)
    ADD "timestamp" WITH TIMESTAMP "UTC"
    STORE IN "mongo_route"
        WITH TABLE "MachineProductionData"

After you deploy this updated Action, all updated data should be stored automatically in the database.

Verify the Complete IoT Automation Pipeline

Monitor Real-Time Data Flow

MQTT Explorer: Use an MQTT client to confirm real-time data publication.

MongoDB Compass: Connect to the database to verify that the data is being stored.

Check Database Storage

Connect to your managed MongoDB database with MongoDB Compass to verify scalable storage:

• Use the connection string from your managed database.
• Navigate to the coreflux-broker-data database, or to the database name you selected.
• Check the MachineProductionData collection for stored documents.

You should see real-time data documents with a structure similar to this:

{
  "_id": {
    "$oid": "68626dc3e8385cbe9a1666c3"
  },
  "energy": 36,
  "energy_wh": 36000,
  "production_status": "active",
  "production_count": 31,
  "stoppage": 0,
  "maintenance_alert": false,
  "timestamp": "2025-06-30 10:58:11",
  "device_name": "station2"
}

As shown earlier, all data remains available in the MQTT broker for other use cases and integrations.

Expand Your Use Case and Integrations

Test Language of Things Capabilities

Publish sample data: Use MQTT Explorer to publish sample datasets to your Coreflux broker. Test different payload structures and different Models or Actions to see how they are processed and stored in MongoDB.

Data validation: Confirm that the data in MongoDB matches the payloads you published. Check consistency and accuracy with MongoDB Compass to make sure your IoT automation integration works as expected. Compare timestamps, field transformations, and data types to validate your real-time data pipeline.

Real-time monitoring: Set up a continuous real-time data feed by using another MQTT data source, such as a simple sensor with MQTT connectivity. Observe how Coreflux and MongoDB handle incoming IoT data streams and evaluate response times for data retrieval and queries.

Build Analytics and Visualizations

Create dashboards: Integrate visualization tools such as Grafana to create dynamic dashboards that display your IoT data, either from live MQTT broker data or from the stored database data. Track metrics such as device uptime, sensor readings, production counts, or maintenance alerts from your automation systems. For monitoring setup guidance, refer to instructions for using Prometheus and Grafana with managed databases.

Trend analysis: Use MongoDB’s aggregation framework to analyze trends over time. Look for patterns, spikes, or anomalies in your real-time data that may indicate system issues or optimization opportunities.

Multi-database integration: Explore integrations with additional managed databases such as PostgreSQL for relational data, MySQL for structured queries, or OpenSearch for advanced analytics and search capabilities. Use Coreflux routes to send data to multiple destinations at the same time.

Optimize and Scale Your IoT Infrastructure

Load testing: Simulate high traffic by publishing many messages at once with LoT Notebook or automated scripts. Monitor how your Coreflux MQTT broker and MongoDB cluster handle the load and identify possible bottlenecks in your data pipeline.

Scaling: Cloud platforms usually provide vertical and horizontal scaling options. Increase virtual machine resources such as CPU, RAM, or storage as your IoT data requirements grow. Scale your managed database cluster to handle larger datasets and configure auto-scaling alerts to notify you when resource limits are being approached.

Frequently Asked Questions

How do I integrate an MQTT broker with MongoDB?

To connect an MQTT broker to MongoDB, set up a Coreflux database route that points to your MongoDB instance. This route uses an authenticated connection string and writes MQTT payloads into MongoDB collections automatically. Coreflux takes care of the connection layer, so you can concentrate on transforming the data with Language of Things syntax.

Can I save MQTT data directly to MongoDB without custom code?

Yes. Coreflux supports this through its MongoDB route functionality. You configure the route with your MongoDB connection string and use a STORE IN directive in your data models. Processed MQTT messages are then written to MongoDB automatically, without custom database integration code. Connection pooling, retries, and error handling are managed by the route.

What are the benefits of using MongoDB with MQTT for IoT applications?

MongoDB works well for IoT scenarios because its document model can handle changing sensor structures and different device formats. Unlike relational databases, it does not require fixed tables in advance. Together with MQTT’s lightweight messaging approach, this creates a practical pipeline for real-time IoT data at scale. Managed MongoDB services can also simplify operations with backups, scaling, and monitoring.

How secure is the connection between Coreflux and MongoDB?

In cloud deployments, private networking such as a VPC can keep communication between Coreflux and MongoDB away from the public internet. TLS can be used to encrypt database connections, and dedicated database users with restricted permissions help follow the principle of least privilege.

Can I use this setup for production IoT deployments?

Yes. This architecture can be used in production when configured properly. Managed databases provide features such as backups, availability options, and monitoring, while Coreflux can scale horizontally for higher message throughput. For production use, configure firewall rules, use strong credentials, enable TLS for MQTT traffic, set up monitoring alerts, and size resources according to the expected message volume.

Conclusion

Connecting a Coreflux MQTT broker to a managed MongoDB service gives you a scalable setup for processing and storing IoT data in real time. In this tutorial, you built an automation pipeline that collects, processes, and saves IoT data efficiently while relying on low-code development methods.

Coreflux provides the scalable messaging layer, while MongoDB offers flexible document-based storage for large volumes of live data. This setup can support use cases such as industrial monitoring, environmental sensor tracking, and smart city infrastructure, helping teams turn incoming IoT data into actionable insights.

By combining the Language of Things Notebook approach with managed cloud services, you now have an IoT- and DevOps-ready foundation that can grow with your requirements. The MQTT broker deployment is prepared for production workloads and can later be expanded with PostgreSQL, MySQL, OpenSearch, or other database technologies.