Draw DAG graph for particular spark program execution.
Senior Data Engineer Interview Questions
2,443 senior data engineer interview questions shared by candidates
Asked questions on Software Development process. Asked questions on Software testing, especially unit testing. What are the challenges you experienced in data transformation with parquet datatype
Nothing particularly very difficult. An open-ended design question followed by a coding question. I felt like I could have give a better answer for the design problem but compared to the rest of the interview that wasn't the worst part.
RDD vs DF - > 1 gb file & 500mb file. What you will use for each one and why
Diff between parquet & CSV and what is the reason for preferring parquet in spark
No dynamic programming-style nonsense. The interviews had been so reasonable that only when I got the offer I recognized how big the company was.
Basic questions, the tech-interviewer didn't seem sure about the expected outcome.
Basic sql using aggregations, window functions like lead,lag, row number etc
1. Coding question (Easy) 2. SQL questions - window functions , query optimisation 3. BigData tools
Q: What is a data lake and how is it compared to a data warehouse?
Viewing 831 - 840 interview questions