hero

Our Network

198
companies
2,037
Jobs

Senior Software Engineer - Spark

Space and Time

Space and Time

Software Engineering
India · Remote
Posted on Wednesday, April 3, 2024

About Space and Time Labs, Inc.

At Space and Time, we are solving Web3’s toughest data analytics challenges at planetary scale with decentralized, peer-to-peer technology. We are building the data foundation of Web3: a fully decentralized and community-operated data warehouse, using novel cryptography to connect tamperproof query results directly to smart contracts. This enables joining trustless on-chain and off-chain data in a single platform, allowing Web3 developers to build new use cases quickly. Our team is growing fast, backed by the top L1 blockchains and VCs, as well as enterprise partners such as Microsoft and Samsung NEXT.
A career at Space and Time is lucrative, fast-paced, and very creative. We value you (and all your ideas), and we bring an endless supply of perks. This includes flexible workweeks + flexible vacation, we attend exciting events/conferences/parties, we’re headquartered in San Clemente (but don’t mind you working remote), and most importantly- we provide analytics technology to the largest dApps, DAOs, DeFi/DEXs, GameFi, NFT platforms, enterprises, etc. We are committed to growing a diverse and welcoming team in a safe space to be yourself and learn from the most innovative minds in blockchain and data warehousing. Help us invent the first truly decentralized data platform!

As a Senior Software Engineer for our Data Platform Engineering team you will join skilled Scala / Spark engineers and core database developers responsible for developing hosted cloud analytics infrastructure (Apache Spark-based), distributed SQL processing frameworks, proprietary data science platforms, and core database optimization. This team is responsible for building the automated, intelligent, and highly performant query planner and execution engines, RPC calls between data warehouse clusters, shared secondary cold storage, etc. This includes building new SQL features and customer-facing functionality, developing novel query optimization techniques for industry-leading performance, and building a database system that's highly parallel, efficient and fault-tolerant. This is a vital role reporting to exec leadership and senior engineering leadership.

Responsibilities:

  • Writing Scala code with tools like Apache Spark + Apache Arrow + Apache Kafka to build a hosted, multi-cluster data warehouse for Web3

  • Developing database optimizers, query planners, query and data routing mechanisms, cluster-to-cluster communication, and workload management techniques

  • Scaling up from proof of concept to “cluster scale” (and eventually hundreds of clusters with hundreds of terabytes each), in terms of both infrastructure/architecture and problem structure

  • Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases to facilitate meta data capturing and management

  • Working with a team of software engineers writing new code to build a bigger, better, faster, more optimized HTAP database (using Apache Spark, Apache Arrow, Kafka, and a wealth of other open source data tools)

  • Interacting with exec team and senior engineering leadership to define, prioritize, and ensure smooth deployments with other operational components

  • Highly engaged with industry trends within analytics domain from a data acquisition processing, engineering, management perspective

  • Understand data and analytics use cases across Web3 / blockchains

Skills & Qualifications

  • 5+ years experience engineering software and data platforms / enterprise-scale data warehouses, preferably with knowledge of open source Apache stack (especially Apache Spark, Apache Arrow, Apache Kafka, Apache Ignite/Geode, and others)

  • 2+ years experience with Scala and Apache Spark (or Kafka)

  • Rock solid engineering fundamentals; query planning, optimizing and distributed data warehouse systems experience is preferred but not required

  • Nice to have: Knowledge of blockchain indexing, web3 compute paradigms, Proofs and consensus mechanisms… is a strong plus but not required

  • Experience with rapid development cycles in a web-based environment

  • Strong scripting and test automation knowledge

Nice to Have's

  • Bachelor’s degree in computer science or related technical field. Masters or PhD a plus.

  • Passionate about Web3, blockchain, decentralization, and a base understanding of how data/analytics plays into this

Notice to Third Party Recruitment/Staffing Agencies:

Space and Time Labs, Inc. does not accept unsolicited resumes or CVs from recruiters or employment agencies without a binding, fully executed, written agreement. Any written or verbal communication with any person within Space and Time Labs, Inc. does not create a binding agreement.

Space and Time Labs, Inc. will not pay a fee to any parties that do not have a valid, fully executed Agreement in place. Any unsolicited resumes or CVs, including those submitted to hiring managers or employees are deemed to be the property of Space and Time Labs, Inc. Space and Time Labs, Inc. explicitly reserves the right to hire those candidate(s) without any financial obligation to the recruiter or agency.

Parties without a fully executed and valid agreement are requested not to contact any hiring managers of the Company with recruiting inquiries or resumes. All candidate resumes must be presented to the hiring managers directly by the Company’s internal Talent Acquisition team.

Agencies interested in partnering with Space and Time Labs, Inc. may contact our Talent Acquisition team (careers@spaceandtime.io).

Our Commitment to Diversity and Inclusion:

At Space and Time, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Space and Time are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics.