Compare This year, we continued hosting Bucharest Big Data meetups at our office, bringing together people passionate about big data and giving them the opportunity to learn from experts in the field. At the beginning of June, we dedicated the evening to real-time data streaming, hybrid environments, and reliable data pipelines.
Speakers & key takeaways
The meetup featured our colleagues, Anamaria Lauric, chapter lead engineer and Cătălin Duță, DevOps engineer, who discussed about enabling the journey from on-premises to cloud through real-time data. The conversation explored how modern data platforms support real-time and near-real-time data flows across different systems, why this matters for analytics and operations, and what it takes to keep data moving smoothly across on-premise and cloud environments.
Later in the evening, Roberto Comșa, data engineer at eSolutions, presented a practical production journey from CDC sources through NiFi, Kafka, Spark and Delta Lake on MinIO. His session focused on why data validation matters in CDC streaming pipelines, with lessons on catching data drift early and building trust in streaming architectures.
Read the key takeaways from Anamaria Lauric and Cătălin Duță:
- A unified, standardised platform replaces fragmented data ingestion approaches | Traditional data ingestion approaches were complex and difficult to manage, even with partial automation. ING’s Data & Analytics platform introduces a unified platform that reduces fragmentation and brings consistency across the entire data flow.
- Real-time data becomes accessible through scalable, cloud-native streaming | With the new streaming capability, data is moved from Kafka to BigQuery in near real-time using Dataflow. This enables a scalable, secure, and efficient way to make data immediately available for analytics.
- The real value comes from standardisation, reuse, and end-to-end automation | Beyond the technology itself, the key benefit is a standardised ‘paved road’ that allows teams to build and run streaming pipelines end-to-end, reducing repetitive work and accelerating time to market
