公司名称

上海微创软件股份有限公司

项目名称

.net

需求描述

岗位职责：

1. Adhere to required privacy standards for data handling of customer content and sensitive information.

2. Maintain existing data pipelines built in Azure Data Lake Analytics (ADLA) and Azure HD Insights (HDI). This includes adding new features to datasets,debugging failures and addressing customer requests / complaints.

3. Migrate data pipelines currently running in ADLA to HDI.

4. Develop new data pipelines in HDI and job orchestration schedules in ADF by leveraging logic from existing data pipelines.

5. Determine optimal schedules and job configuration parameters (memory/disk) before launching these pipelines to production.

6. Build telemetry and dashboards to monitor health of new/existing data pipelines and dataset quality.

7. Provide business-hours on-call support for current data pipelines running in production.

任职要求：

1. 2+ years' experience in designing, implementing and debugging software.

2. Clear and effective communication skills.

3. Experience working in an agile environment with devops tooling for work item tracking (such as ADO).

3. Experience in development, debugging and performance characteristics of data pipelines, running on Azure HDInsight Spark.

4. Programming in C# or hadoop, spark.

5. Experience with Azure Data Factory for data movement and job orchestration.