Data Engineer

4 weeks ago


Beijing, China Unison Consulting Pte Ltd Full time
  • Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data
  • Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data
  • Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch
  • Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions
  • Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch
  • Develop and maintain scalable and fault-tolerant Spark applications, adhering to industry best practices and coding standards
  • Troubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration

Requirements

  1. 5+ Years as Data Engineer
  2. 5 Years experience in Spark, Scala, Elastic Search
  3. Develop and maintain scalable and fault-tolerant Spark applications, adhering to industry best practices and coding standards
  4. Monitor and analyze job performance metrics, identify bottlenecks, and propose optimizations in both Spark and Elasticsearch components
  5. Stay updated with emerging trends and advancements in the big data technologies space to ensure continuous improvement and innovation

  • Perfect Engineer

    4 weeks ago


    Beijing, China Data Annotation Full time

    The position Mobileye's Perfect Engineer should understand Mobileye's map products and technologies, understand targeted technical issues raised by HQ and customers, collect, and organize relevant data, and continuously optimize the accuracy and precision of REM products. What will your job look like: Identify production map bug/issue and fix...


  • Beijing, China Unison Consulting Pte Ltd Full time

    Develop and implement data pipelines for ingesting and collecting data from various sources into a centralized data platform. Develop and maintain ETL jobs using AWS Glue services to process and transform data at scale. Optimize and troubleshoot AWS Glue jobs for performance and reliability. Utilize Python and PySpark to efficiently handle large volumes of...

  • Data Engineer

    4 weeks ago


    Beijing, China Unison Consulting Pte Ltd Full time

    Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions Ensure data quality and integrity throughout the data processing lifecycle Design and...

  • Software Engineer

    1 month ago


    Beijing, China EDB Full time

    A Little About UsInnovative, collaborative minds wanted. The world loves Postgres. We envision a world where organizations thrive by harnessing the full power of Postgres, the world’s fastest growing and most loved and used open source database. Our mission is to enable data teams everywhere to harness the full power of Postgres, whether on premises or in...


  • Beijing, China Zenni Optical Full time

    Zenni was founded on the idea that everyone who needs glasses should be able to afford high-quality frames and top-of-the-line lenses. By selling directly from our factories to our customers through our website we save them money and offer a huge selection of options. We have state-of-the-art production facilities and advanced prescription edging labs that...


  • Beijing, China Zenni Optical Full time

    Zenni was founded on the idea that everyone who needs glasses should be able to afford high-quality frames and top-of-the-line lenses. By selling directly from our factories to our customers through our website we save them money and offer a huge selection of options. We have state-of-the-art production facilities and advanced prescription edging labs that...

  • Senior Data Scientist

    4 weeks ago


    Beijing, China Advance Intelligence Group Full time

    1. Working experiences in credit risk management;2. Good English skills, fluent in business communication;3. Good knowledge and understanding in risk management process, experiences in hands-on developing risk control model is a plus;4. Knowledge and clear expression in regular risk management modelling;5. Good communication skills and teamwork...

  • Data Science Intern

    4 weeks ago


    Beijing, China WorldQuant Full time

    WorldQuant develops and deploys systematic financial strategies across a variety of asset classes and global markets. We seek to produce high-quality predictive signals (alphas) through our proprietary research platform to employ financial strategies focused on market inefficiencies. Our teams work collaboratively to drive the production of alphas and...


  • Beijing, China Microsoft Full time

    Overview Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where...


  • Beijing, China Microsoft Full time

    Overview We are inviting you to join the Mobile Experiences & Platform Team (MXP). The Mobile Experiences & Platform Team (MXP) is building the future. We are a world class platform, middleware, apps, and services organization that brings end-to-end experiences to life on a diverse range of form factors - empowering customers through powerful...


  • Beijing, China NTT DATA Full time

    Job Description...


  • Beijing, China Data Annotation Full time

    The position ...


  • Beijing, China HITACHI ENERGY (CHINA) LTD. Full time

    Description : Mission statement The quality and innovation of the engineering in our products, systems, and service offerings keep our customers ahead. Developing solutions for our customers all over the world and working on our market-leading technologies will challenge and enrich your work and your mind. We are looking for candidates for the...


  • Beijing, China Amazon Connect Technology Services (Beijing) Co., Ltd. Full time

    Amazon Web Services is growing rapidly, and we are looking for Data Center Technical Support Engineering Operations Technician (EOTs) Interns to join our expanding Infrastructure Operations Technical Support team. This committed group works to provide essential technical support to the physical infrastructure that supports Amazon Web Services.Specifically,...


  • Beijing, China Microsoft Full time

    Overview Online Advertising is one of the fastest growing businesses on the Internet today. Microsoft is innovating rapidly to grow its share of this market by providing the advertising industry with a world-class online advertising platform and service. The Bing Ads team is responsible for designing, developing, and maintaining a diverse family of...


  • Beijing, China Amazon (China) Holding Company Limited - D24 Full time

    Are you interested in applying your strong quantitative analysis and big data skills to world-changing problems? Are you interested in driving the development of methods, models and systems for strategy planning, transportation and fulfillment network? Are you interested to cooperate with Amazonians around the world? If so, then this is the job for you. Our...

  • Principal Engineer

    4 weeks ago


    Beijing, China Amazon Information Service (Beijing) Co., Ltd. - A67 Full time

    Principal Engineers provide technical leadership at Amazon.com. They help establish technical standards and drive Amazon’s overall technical architecture, engineering practices, and engineering methodologies. They work on our hardest problems, building high quality, architecturally sound systems that are aligned with our business needs. They think globally...


  • Beijing, China SAP Full time

    We help the world run better Our company culture is focused on helping our employees enable innovation by building breakthroughs together. How? We focus every day on building the foundation for tomorrow and creating a workplace that embraces differences, values flexibility, and is aligned to our purpose-driven and future-focused work. We offer a highly...


  • Beijing, China Algorithms Full time

    The position Mobileye is the global leader in the development of computer vision and machine learning, data analysis, localization and mapping for Advanced Driver Assistance Systems and autonomous driving. Mobileye’s technology helps keep passengers safer on the roads, reduces the risks of traffic accidents, saves lives and has the potential to...

  • QA Engineer

    7 days ago


    Beijing, China Veeva Systems Full time

    The Role China SFA is a SaaS CRM product defined and built specifically in China, for China. We are looking for functional QA/Senior QA Engineers with BlackBox and automation testing experience to join the team. What You’ll Do Responsible for functional testing on the web, Enterprise WeChat, and iOS app Work closely with product managers and...