Data Engineer with Python

4 weeks ago


Beijing, China Unison Consulting Pte Ltd Full time

  • Develop and implement data pipelines for ingesting and collecting data from various sources into a centralized data platform.
  • Develop and maintain ETL jobs using AWS Glue services to process and transform data at scale.
  • Optimize and troubleshoot AWS Glue jobs for performance and reliability.
  • Utilize Python and PySpark to efficiently handle large volumes of data during the ingestion process.
  • Design and implement scalable data processing solutions using PySpark to transform raw data into a structured and usable format.
  • Apply data cleansing, enrichment, and validation techniques to ensure data quality and accuracy.
  • Create and maintain ETL processes using Python and PySpark to move and transform data between different systems.
  • Optimize ETL workflows for performance and efficiency.
  • Collaborate with data architects to design and implement data models that support business requirements.
  • Ensure data structures are optimized for analytics and reporting.
  • Work with distributed computing frameworks, such as Apache Spark, to process and analyze large-scale datasets.
  • Manage and optimize databases, both SQL and NoSQL, to support data storage and retrieval needs.
  • Implement indexing, partitioning, and other database optimization techniques.
  • Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver effective solutions.
  • Work closely with software engineers to integrate data solutions into larger applications.
  • Implement monitoring solutions to track data pipeline performance and proactively identify and address issues
  • Ensure compliance with data privacy regulations and company policies.
  • Stay abreast of industry trends and advancements in data engineering, Python, and PySpark.

Requirements

  • Proficiency in Python and PySpark.
  • Strong knowledge of data engineering concepts and best practices.
  • Hands-on experience with AWS Glue and other AWS services.
  • Experience with big data technologies and distributed computing.
  • Familiarity with database management systems (SQL and NoSQL).
  • Understanding of ETL processes and data modeling.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration skills.
  • Bachelor's degree in Computer Science, Information Technology, or a related field.


  • Beijing, China Software Full time

    The position Mobileye China R&D Center is looking for highly motivated AV Map Validation Engineer to make huge impact on the safety of advanced driving assistance systems (ADAS) and fully autonomous vehicles (AV).AV Map Validation Engineer at Mobileye have solid mathematical and algorithmic foundation in specific domain of graph, computer vision,...

  • Software Engineer

    1 month ago


    Beijing, China EDB Full time

    A Little About UsInnovative, collaborative minds wanted. The world loves Postgres. We envision a world where organizations thrive by harnessing the full power of Postgres, the world’s fastest growing and most loved and used open source database. Our mission is to enable data teams everywhere to harness the full power of Postgres, whether on premises or in...

  • Perfect Engineer

    4 weeks ago


    Beijing, China Data Annotation Full time

    The position Mobileye's Perfect Engineer should understand Mobileye's map products and technologies, understand targeted technical issues raised by HQ and customers, collect, and organize relevant data, and continuously optimize the accuracy and precision of REM products. What will your job look like: Identify production map bug/issue and fix...


  • Beijing, China Ericsson Full time

    Description Join our Team About this opportunity We are looking for a Digital Hardware Design Intern  What you will do Be responsible for Digital Hardware design work in CCC Hardware Development Project. You will bring • Master or above degree and major in electrical engineering and related fields. • Enthusiastic about AI with a...


  • Beijing, China Ericsson Full time

    Description Join our Team About this opportunity We are looking for a Digital Hardware Design Intern  What you will do Be responsible for Digital Hardware design work in CCC Hardware Development Project. You will bring • Master or above degree and major in electrical engineering and related fields. • Enthusiastic about AI with a...


  • Beijing, China Ericsson Full time

    Description Join our Team About this opportunity We are looking for a Digital Hardware Design Intern  What you will do Be responsible for Digital Hardware design work in CCC Hardware Development Project. You will bring • Master or above degree and major in electrical engineering and related fields. • Enthusiastic about AI with a...


  • Beijing, China Ericsson Full time

    Description Join our Team About this opportunity We are looking for a Digital Hardware Design Intern  What you will do Be responsible for Digital Hardware design work in CCC Hardware Development Project. You will bring • Master or above degree and major in electrical engineering and related fields. • Enthusiastic about AI with a...


  • Beijing, China Zenni Optical Full time

    Zenni was founded on the idea that everyone who needs glasses should be able to afford high-quality frames and top-of-the-line lenses. By selling directly from our factories to our customers through our website we save them money and offer a huge selection of options. We have state-of-the-art production facilities and advanced prescription edging labs that...


  • Beijing, China Zenni Optical Full time

    Zenni was founded on the idea that everyone who needs glasses should be able to afford high-quality frames and top-of-the-line lenses. By selling directly from our factories to our customers through our website we save them money and offer a huge selection of options. We have state-of-the-art production facilities and advanced prescription edging labs that...

  • Data Science Intern

    4 weeks ago


    Beijing, China WorldQuant Full time

    WorldQuant develops and deploys systematic financial strategies across a variety of asset classes and global markets. We seek to produce high-quality predictive signals (alphas) through our proprietary research platform to employ financial strategies focused on market inefficiencies. Our teams work collaboratively to drive the production of alphas and...

  • Data Modeler

    4 weeks ago


    Beijing, China DBS Bank Full time

    Data Modeler-(WD56744) Responsibilities · Design data mart solutions in big data platform to store and retrieve company information · Analyze requirements for new requests on data mart enhancement · Migrate data analysis process from legacy systems to big data platform · Design conceptual and logical data models and flowcharts · Improve system...

  • Data Engineer

    4 weeks ago


    Beijing, China Unison Consulting Pte Ltd Full time

    Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions Ensure data quality and integrity throughout the data processing lifecycle Design and...


  • Beijing, China Microsoft Full time

    Overview We are inviting you to join the Mobile Experiences & Platform Team (MXP). The Mobile Experiences & Platform Team (MXP) is building the future. We are a world class platform, middleware, apps, and services organization that brings end-to-end experiences to life on a diverse range of form factors - empowering customers through powerful...


  • Beijing, China DBS Bank Full time

    Data Enablement Specialist-(WD55058) Main Responsibility: Data enablement team’s mission is to prompt data usage in ADA lite platform. During this transformation journey, this position needs to take up product manager role to engage with BU/SU for 1) collecting their data requirements; 2) proposing executable solution; 3) collaborating with tech team and...

  • Data Scientist

    4 weeks ago


    Beijing, China Trexquant Investment Full time

    As a member of the Data team at Trexquant, you will be involved in parsing and analyzing large data sets, working on discovering and obtaining new sources of data, and collaborating with the Alpha and Strategy Research team to build predictive machine learning models.Responsibilities Explore and learn about a wide range of data sets that are used to develop...

  • Data Engineer

    4 weeks ago


    Beijing, China Unison Consulting Pte Ltd Full time

    Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in...

  • QA Engineer

    7 days ago


    Beijing, China Veeva Systems Full time

    The Role China SFA is a SaaS CRM product defined and built specifically in China, for China. We are looking for functional QA/Senior QA Engineers with BlackBox and automation testing experience to join the team. What You’ll Do Responsible for functional testing on the web, Enterprise WeChat, and iOS app Work closely with product managers and...


  • Beijing, China Algorithms Full time

    The position Mobileye is the global leader in the development of computer vision and machine learning, data analysis, localization and mapping for Advanced Driver Assistance Systems and autonomous driving. Mobileye’s technology helps keep passengers safer on the roads, reduces the risks of traffic accidents, saves lives and has the potential to...

  • Digitization engineer

    4 weeks ago


    Beijing, China HITACHI ENERGY HIGH VOLTAGE SWITCHGEAR CO., LTD., BEIJING Full time

    Description : Your responsibilities: You will identify the opportunities of digitalization by collecting and analyzing requirements from functions and transform them into digital project base on technology feasibility study and investment vs. benefit analysis. You will regularly review the operation status of digital system and continuous to...

  • Senior Data Scientist

    4 weeks ago


    Beijing, China Advance Intelligence Group Full time

    1. Working experiences in credit risk management;2. Good English skills, fluent in business communication;3. Good knowledge and understanding in risk management process, experiences in hands-on developing risk control model is a plus;4. Knowledge and clear expression in regular risk management modelling;5. Good communication skills and teamwork...