Round 1: a company introduction and Q/A Round 2: Tech task demo, OOP, ACID, CAP theorem, Java Round 3: Design a highly loaded system with details, data models, and prod-ready diagram in 120 min
Lead Data Engineer Interview Questions
208 lead data engineer interview questions shared by candidates
Basic programming data structures,sql distributed data computing using spark.
Architecture of data engineer problems.
Talked lots about background and experience. Lots of questions around Snowflake's architecture.
Write APIs to calculate percentile values from a batch of requests of integer values
What do you know about this company?
Please prepare an architectural diagram of a data platform you have been involved in and you feel is relevant to Outra. In the interview you will be asked to talk through the various components, the technical choices, potential issues and any recommendations for the future.
1) Project explanation 2) Spark Optimization 3) Data quality process and ways to handle it 4) Questions on Databricks, Datafactory and few services from Azure mentioned in the CV 5) The whole interview was a discussion rather than just one way process which made me very comfortable
to write SQL queries and Pyspark coding to load, filter, aggregate and save as another new table, strategies to design ETL validations to merge to consume kafka and s3 bucket files. Strategies to consume data from s3 bucket focusing on spark architecture, designing clusters, shuffling, narrow/wide transformations, authenticating Azure data lakes, resource manager in spark.
Is it possible to assign a specific broker to a Kafka producer?
Viewing 171 - 180 interview questions