Big data infrastructure with HPC storage for high IO load and data streaming
Running Kafka, NATS or RabbitMQ yourself: reliably transmitting event streams between systems – with storage and computing power that can keep up. Stable under parallel access and sustained load, S3-compatible big data and AI workflows – for scalable data volumes and high-performance data processing, from data capture and integration to analysis with AI and machine learning.
- Stable under parallel access and sustained load – consistently high IOPS and stable latency so that computing resources reach their full potential.
- S3-compatible big data and AI workflows – scalable data platforms, data lakes and API-based data access.
- For scalable data volumes and high-performance processing – Worker Pool, Power Pool or High Availability, Kubernetes in seconds.
- Managed servers for demanding workloads – monitoring, performance optimisation, scaling adjustments and direct technical contacts.
Performance architecture for demanding big data workloads
A high-performance big data infrastructure consists of several layers: the hosting environment forms the technical basis for stable operation, compute resources deliver the processing power for analytics, streaming and HPC, and the storage architecture determines how efficiently large data volumes are processed, stored and served in parallel.
ccloud³ virtual machines
Worker Pool for light to moderate load, Power Pool for high CPU demands, High Availability for maximum resilience – start a data streaming VM today.
centron Kubernetes
Create a Kubernetes cluster in seconds and use the power of containerisation to scale streaming and analytics applications with ease.
S3 Object Storage
Store large volumes of data in an S3-compatible way – scalable data platforms, data lakes and flexible, API-based data access as it is common in big data and AI environments.
Volumes Block Storage
NVMe volumes for IO-intensive workloads with high parallelism: stable, predictable performance under load and additional SSD capacity when needed.
Use cases for big data
The cloud infrastructure was designed specifically to map these use cases – with hosting as the technical foundation, compute resources for analysis, streaming and HPC processes, and a storage architecture that processes, stores and delivers large data volumes efficiently and in parallel.
Data analysis
Processing large volumes of structured and unstructured data – raw data from IoT systems, log files, transaction systems or sensors as the basis for analyses, machine learning models and real-time evaluations; the stack for it on data analytics.
Data streaming & real-time processing
Continuous processing of data streams with stable performance under high IO load – streaming frameworks such as Apache Kafka, Spark or Flink on ccloud³ VMs, Kubernetes clusters or dedicated servers.
AI and machine learning workloads
Training and inference with high data access and parallel processing – on GPU-optimised servers with an S3 data lake and NVMe volumes; details on LLM inference.
HPC and simulation environments
Compute-intensive workloads with predictable storage and throughput performance – HPC servers and block storage that delivers stable IOPS under parallel access.
Consistently high IOPS and stable latency
When analysing large data volumes, running HPC applications or processing data streams, consistently high IOPS and stable latency are what count – even powerful compute resources only realise their potential if the storage keeps up. Block storage delivers predictable performance under parallel access, and S3-compatible object storage carries scalable data platforms. And instead of operating the platform yourself, you can opt for a managed server model with technical responsibility for operations.
- Data analysis – processing large volumes of structured and unstructured data.
- Data streaming & real-time processing – continuous processing of data streams with stable performance under high IO load.
- AI and machine learning workloads – training and inference with heavy data access and parallel processing.
- HPC and simulation environments – compute-intensive workloads with predictable storage and throughput performance.
The right centron products
Customers typically implement big data and streaming workloads with these modules – combinable and extensible at any time.
- Worker, Power & HA Pool
- Linux & Windows
- Billed by the hour
- Stable IOPS under load
- Can be expanded whilst in operation
- Snapshot-compatible
- S3-compatible API
- Free traffic
- Unlimited scalability
How much does data streaming cost at centron?
Data streaming infrastructure: Run Kafka and event pipelines on your own VMs – low latency, full data sovereignty. The cornerstone is ccloud³ VMs from €3.11 per month – billed by the hour, with no minimum contract term. This is supplemented as required by Volumes and Kubernetes. The infrastructure runs in our own German data centres, which are certified to ISO 27001 on the basis of IT-Grundschutz and hold an unrestricted BSI C5:2020 Type 1 attestation. New accounts receive a €200 starting credit.
| Building block | Price |
|---|---|
| ccloud³ VMs | from €3.11 per month |
| Volumes | from €0.05 per GB per month |
| Kubernetes | from €29.99 per month |
Frequently asked questions
What are typical requirements of big data workloads?
How do I process raw data or real-time data efficiently?
Can I rent servers from centron – including for Apache Kafka or Spark?
Which storage option is suitable for large data volumes?
Why is storage performance so decisive for big data?
Where is the data stored?
How does centron differ from classic cloud providers?
Can the environment be operated in-house or fully managed?
Which streaming platforms can I run?
How does a broker cluster stay fail-safe?
Can this be implemented in a way that complies with the GDPR?
More on Data & Storage
Get started for free
Sign up and receive €200 credit at centron within your first 60 days.
This promotional offer applies to new accounts only. Available exclusively to businesses.