Real-Time Data Integration: Build and optimize real-time data pipelines using Kafka, Spark, and Delta Live Tables. o Ensure the existence of, or adaptation to, full Data Life Cycle Management (DLCM) and end-to-end testing. * Knowledge Sharing: the team on best practices, modern tools (e.g., Databricks, Snowflake, AI adaptation and integrations like Cursor/CodeRabbit), and cloud-native scalability. * Familiarity in AI/ML pipelines and MLOps (e.g., MLflow, feature stores) and complex system integration within modern data technologies.
mehr