Big-data platform operations development engineer-N

Found in: Talent CN S2 - 6 days ago


Beijing, China TD SYNNEX Full time

岗位职责:

- 负责设计和构建大数据平台的各类组件,包括但不限于数据采集,计算,对外服务及治理平台的设计和开发搭建,降低数据的使用门槛,实现数据的最大价值;

- 基于开源解决方案,构建完整的数据开发、治理框架,以满足实时、近实时以及批量计算的需求。

- 持续升级和解决开源产品中的问题,包括Spark、Hive、Hdfs、Hudi、StarRocks中的bug修复,基于源码的新功能开发和集成。

- 探索和实践新技术和现有平台的集成。

 

技能和要求:

- 计算机相关专业,本科及以上学历。4年以上大数据相关工作经验。

- 精通一种或多种大数据相关组件技术,包括但不限于 Hadoop、Spark、Hive、Flink、Debezium、Canal、ELK、Clickhouse、Starrocks(doris)、调度系统、元数据管理系统、搜索引擎技术等,理解其内部原理和适用场景,了解核心源码及调优方法。精通Spark的优先考虑。

- 精通Java,基础扎实 ,对于多线程、网络、IO方面有实际的应用经验,了解Java 内存模型,垃圾回收机制,对于Java的内存监控、调优、线上Debug有丰富经验。

- 有大数据中台建设经验、数据治理经验者优先。

- 熟悉Yarn、Kubernetes等调度和资源管理系统,参与过大型集群的搭建或维护优先。

- 熟悉Azure , AWS, GCP的优先。

- 需要熟练的英文读写能力,口语能力好的优先考虑。

What’s In It For You?

  • Elective Benefits: Our programs are tailored to your country to best accommodate your lifestyle.
  • Grow Your Career: Accelerate your path to success (and keep up with the future) with formal programs on leadership and professional development, and many more on-demand courses.
  • Elevate Your Personal Well-Being: Boost your financial, physical, and mental well-being through seminars, events, and our global Life Empowerment Assistance Program.
  • Diversity, Equity & Inclusion: It’s not just a phrase to us; valuing every voice is how we succeed. Join us in celebrating our global diversity through inclusive education, meaningful peer-to-peer conversations, and equitable growth and development opportunities.
  • Make the Most of our Global Organization: Network with other new co-workers within your first 30 days through our onboarding program.
  • Connect with Your Community: Participate in internal, peer-led inclusive communities and activities, including business resource groups, local volunteering events, and more environmental and social initiatives.


  • Big-data platform operations development engineer-N

    Found in: Talent CN S2 - 6 days ago


    Beijing, China TD SYNNEX Full time

    岗位职责:- 负责设计和构建大数据平台的各类组件,包括但不限于数据采集,计算,对外服务及治理平台的设计和开发搭建,降低数据的使用门槛,实现数据的最大价值;- 基于开源解决方案,构建完整的数据开发、治理框架,以满足实时、近实时以及批量计算的需求。-...

  • Big-data platform operations development engineer-N

    Found in: beBee S CN - 2 weeks ago


    Beijing, China TD SYNNEX CH Full time

    岗位职责:- 负责设计和构建大数据平台的各类组件,包括但不限于数据采集,计算,对外服务及治理平台的设计和开发搭建,降低数据的使用...

  • Data Modeler

    Found in: Talent CN S2 - 6 days ago


    Beijing, China DBS Bank Full time

    Data Modeler-(WD56744) Responsibilities · Design data mart solutions in big data platform to store and retrieve company information · Analyze requirements for new requests on data mart enhancement · Migrate data analysis process from legacy systems to big data platform · Design conceptual and logical data models and flowcharts · Improve system...

  • Senior Designer- Microsoft Fabric platform

    Found in: Talent CN S2 - 3 days ago


    Beijing, China Microsoft Full time

    Overview Microsoft is a company where passionate innovators come to collaborate, envision what can be and take their careers further. This is a world of more possibilities, more innovation, more openness, and the sky is the limit thinking in a cloud-enabled world. Microsoft’s Azure Data engineering team is leading the transformation of analytics in...

  • Data & Applied Scientist- Mobile Experiences & Platform

    Found in: Talent CN S2 - 6 days ago


    Beijing, China Microsoft Full time

    Overview We are inviting you to join the Mobile Experiences & Platform Team (MXP). The Mobile Experiences & Platform Team (MXP) is building the future. We are a world class platform, middleware, apps, and services organization that brings end-to-end experiences to life on a diverse range of form factors - empowering customers through powerful...

  • Data Engineer with Python

    Found in: Talent CN S2 - 6 days ago


    Beijing, China Unison Consulting Pte Ltd Full time

    Develop and implement data pipelines for ingesting and collecting data from various sources into a centralized data platform. Develop and maintain ETL jobs using AWS Glue services to process and transform data at scale. Optimize and troubleshoot AWS Glue jobs for performance and reliability. Utilize Python and PySpark to efficiently handle large volumes of...

  • Software Development Engineer, Automated Marketing

    Found in: Talent CN S2 - 3 days ago


    Beijing, China Amazon (China) Holding Company Limited Full time

    Amazon’s Automated Marketing & Events organization is building the Internet’s largest-scale Search Marketing systems worldwide. The team is responsible for a number of business-critical systems that generates, target, measures and optimizes hundreds of millions of search engine ad placements and organic search listings (SERPs). We bring prospective...

  • Machine Learning Engineer

    Found in: Talent CN S2 - 6 days ago


    Beijing, China Zenni Optical Full time

    Zenni was founded on the idea that everyone who needs glasses should be able to afford high-quality frames and top-of-the-line lenses. By selling directly from our factories to our customers through our website we save them money and offer a huge selection of options. We have state-of-the-art production facilities and advanced prescription edging labs that...

  • Machine Learning Engineer

    Found in: Talent CN S2 - 6 days ago


    Beijing, China Zenni Optical Full time

    Zenni was founded on the idea that everyone who needs glasses should be able to afford high-quality frames and top-of-the-line lenses. By selling directly from our factories to our customers through our website we save them money and offer a huge selection of options. We have state-of-the-art production facilities and advanced prescription edging labs that...

  • Development Operations Engineer

    Found in: Talent CN S2 - 6 days ago


    Beijing, China Clarivate Full time

    We are looking for a Development Operations Engineer to join our Cloud team in Beijing. This is an amazing opportunity to work on Ex Libris, our SaaS solutions designed for libraries. The team consists of 6 people and is reporting to the Manager, Development Operations Engineering. We have a great skill set in Cloud and we would love to speak with you if...


  • Beijing, China Amazon (China) Holding Company Limited Full time

    Amazon Global Mile Cross Border Fulfillment (XBF) team is focused on breaking down barriers to allow 3rd party sellers to sell their items to customers around the world. The XBF team develops software that removes friction from the process of cross border selling for 3rd party sellers. We need your help to grow the business by building highly-available,...

  • Software Engineer

    Found in: Talent CN S2 - 2 weeks ago


    Beijing, China EDB Full time

    A Little About UsInnovative, collaborative minds wanted. The world loves Postgres. We envision a world where organizations thrive by harnessing the full power of Postgres, the world’s fastest growing and most loved and used open source database. Our mission is to enable data teams everywhere to harness the full power of Postgres, whether on premises or in...


  • Beijing, China Amazon (China) Holding Company Limited Full time

    Amazon Global Mile Cross Border Fulfillment (XBF) team is focused on breaking down barriers to allow 3rd party sellers to sell their items to customers around the world. The XBF team develops software that removes friction from the process of cross border selling for 3rd party sellers. We need your help to grow the business by building highly-available,...

  • Data Engineer

    Found in: Talent CN S2 - 6 days ago


    Beijing, China Unison Consulting Pte Ltd Full time

    Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in...

  • Senior Software Engineer(Ads)

    Found in: Talent CN S2 - 12 hours ago


    Beijing, China Microsoft Full time

    Overview Online Advertising is one of the fastest growing businesses on the Internet today. Microsoft is innovating rapidly to grow its share of this market by providing the advertising industry with a world-class online advertising platform and service. The Bing Ads team is responsible for designing, developing, and maintaining a diverse family of...

  • Data Engineer

    Found in: Talent CN S2 - 6 days ago


    Beijing, China Unison Consulting Pte Ltd Full time

    Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions Ensure data quality and integrity throughout the data processing lifecycle Design and...

  • Senior Software Dev Engineer

    Found in: Talent CN S2 - 12 hours ago


    Beijing, China Amazon (China) Holding Company Limited Full time

    The Role:As a Senior Software Development Engineer(SDE) – Platform, you will engage with an experienced cross-disciplinary staff to conceive and design innovative consumer products. You must be responsive, flexible and able to succeed within an open collaborative peer environment. You will work closely with the hardware, platform, framework, dev ops,...

  • Applied Scientist, Analytics Technology and Engineering

    Found in: Talent CN S2 - 6 days ago


    Beijing, China Amazon (China) Holding Company Limited - D24 Full time

    Are you interested in applying your strong quantitative analysis and big data skills to world-changing problems? Are you interested in driving the development of methods, models and systems for strategy planning, transportation and fulfillment network? Are you interested to cooperate with Amazonians around the world? If so, then this is the job for you. Our...

  • IT Intern

    Found in: beBee S CN - 4 weeks ago


    Beijing, China Cummins Inc. Full time

    IT Intern - Data Engineer Description 协助架构师进行工具调研,辅助查看工具脚本 进行多个网站工具对比输出各工具优缺点 熟悉Apache网站开源项目规则 支持终端用户进行数据上传/传输等 Qualifications 思维活跃、积极主动、认真负责 熟悉技术语言:Python & Java 熟悉AI /...

  • Software Engineer 2- Mobile Experiences

    Found in: Talent CN S2 - 6 days ago


    Beijing, China Microsoft Full time

    Overview We are inviting you to join the Mobile Experiences & Platform Team (MXP). The Mobile Experiences & Platform Team (MXP) is building the future. We are a world class platform, middleware, apps, and services organization that brings end-to-end experiences to life on a diverse range of form factors - empowering customers through powerful...