Apply with hirly
PySpark Developer
Infosys · Bangalore, India
Upload your resume to see how well you match this job — free, in seconds, no account needed.
Your resume is used only to score it against this job. If you don't create an account, it is deleted within 24 hours.
Responsibilities Key Responsibilities Develop and maintain data pipelines using PySpark Process and analyze large-scale datasets in distributed environments Design and implement ETL/ELT workflows Optimize Spark jobs for performance and scalability Work with data stored in HDFS, Hive, or cloud storage (S3, ADLS) Collaborate with data engineers, analysts, and business teams Ensure data quality, integrity, and governance Debug and troubleshoot data processing issues Automate workflows using scheduling tools (Airflow, Oozie, etc.) Write clean, scalable, and efficient code Required Skills & Qualifications Technical Skills Strong proficiency in Python and PySpark Good experience with Apache Spark (RDDs, DataFrames, Spark SQL) Knowledge of Hadoop ecosystem (HDFS, Hive) Experience in ETL pipeline development Familiarity with SQL and database concepts Experience with data formats (Parquet, ORC, JSON, CSV) Basic understanding of distributed computing concepts Exposure to version control tools (Git) Technical requirements Technology->Big Data
- Data Processing->PySpark Education MCA,MSc,MTech,Bachelor of Engineering,BCA,BSc,BTech