Software Development Engineer in Test - GenAI (ROU, Remote)

Undelucram.ro on behalf of: Crowdstrike SRLAbout the Role: The GenAI Platform team builds the core building blocks powering CrowdStrike’s next generation of AI products—model inference, knowledge bases, agents/tools, guardrails, cost management, and the SDLC around AI artifacts (evaluation, benchmarking, versioning, deploy). This platform is central to CrowdStrike’s AI strategy and supports our path to $10B ARR. You’ll partner closely with the Charlotte AI team, data scientists, and engineering/product groups across CrowdStrike, and engage with ProdSec, InfoSec, Legal, and Privacy to deliver secure, compliant, enterprise‑grade AI capabilities. What You’ll Do:

• Design and build automation for services, data pipelines, and LLM workflows using Go/Python—covering unit, integration, contract, end‑to‑end, and performance tests.
• Create LLM evaluation pipelines: curate datasets, define ground truth, automate scoring, and set release gates for accuracy, safety, latency, and cost; support human‑in‑the‑loop review.
• Define test strategy and coverage for functional and non‑functional needs (performance, scalability, resilience, stability), including Kafka‑based async flows and OpenSearch/SQL/Redis data layers.
• Harden safety and guardrails: test for prompt injection, jailbreaks, PII leakage, toxicity, hallucinations, and adherence to prompt/tool contracts.
• Shift left in CI/CD: integrate fast/slow suites, flaky‑test management, coverage and quality gates in Jenkins/Git workflows; deliver build health dashboards.
• Own observability for quality: instrument tests and services (metrics, logs, traces) and track SLIs/SLOs such as p95 latency, error rate, and cost per request/token.
• Validate integrations from internal customers with the GenAI Platform SDKs/APIs; provide reference fakes/mocks and test harnesses.
• Participate in design and code reviews with a strong focus on testability, reliability, and security; mentor engineers on quality best practices.
• Collaborate cross‑functionally (ProdSec/InfoSec/Legal/Privacy) to meet compliance, data residency, auditability, and RBAC requirements.
• Continuously improve tools, frameworks, and processes; stay current with testing techniques and GenAI evaluation methods.

What You’ll Bring (Minimum Qualifications):

• 5+ years building test automation as an SDET or software engineer or equivalent practical experience.
• Proficiency in Go or Python (ideally both), writing production‑quality code and tests.
• Solid CS fundamentals (data structures, algorithms, OS, networking, distributed systems).
• Hands-on experience with automated testing frameworks (e.g., pytest, Go test/Ginkgo/Gomega, JUnit/TestNG); API (REST/gRPC) and message‑driven testing.
• CI/CD expertise (Jenkins, Git/Bitbucket), artifact/version management, and branch strategies.
• AWS experience (S3, EC2, IAM; familiarity with VPC networking and security basics).
• Experience testing AI/ML or LLM‑integrated systems (evaluation datasets, prompt/version management, safety/guardrails).
• Strong collaboration and clear written/verbal communication; comfortable operating with ambiguity and iterating quickly.

Nice to Have:

• GenAI in production: LLM serving, RAG, agents/tools, guardrails/safety, evaluation frameworks; tuning for cost/latency; vector search (OpenSearch k‑NN/FAISS).
• Familiarity with frameworks like LangChain, LlamaIndex, CrewAI, n8n.
• Kubernetes/containers and IaC (Docker, Terraform/CloudFormation).
• Observability (OpenTelemetry), UI automation (Playwright), contract testing (e.g., Pact).
• Cybersecurity domain knowledge and secure SDLC; experience with privacy/compliance in enterprise environments.