Pyspark Developer

Job Description:

• Developer must have sound knowledge in Apache Spark and Python programming.
• Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.
• Ability to design, build and unit test the application in Spark/Pyspark.
• In-depth knowledge of Hadoop, Spark, and similar frameworks.
• Ability to understand existing ETL & logic to convert into Spark/PySpark/ Spark SQL.
• Knowledge of Unix shell scripting, RDBMS, Hive, HDFS File System, HDFS File Types, HDFS compression codec.
• Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources.

• ·    Payments Industry Background

  ·    Hadoop and Bigdata technologies cloud exposure

Skillety is a gig-economy startup crowdsourcing platform. We provide various types of Staffing services like Full-Time, Part-time, Contractors & Freelancers. We help our Clients to move away from the old and soiled hierarchical business structure, and evolve into a 21st Century on-demand, open talent, cross-functional team; with a skilled and passionate workforce who are more engaged, effective & productive.