Resume Score
CV/Résumé Score
  • Expertini Resume Scoring: See how well your CV/Résumé matches this job: Python Systems Engineer – LLM Evaluation.
Vellore | Expertini

Urgent! Python Systems Engineer – LLM Evaluation Job | Supercoder

Python Systems Engineer – LLM Evaluation



Job description

Greetings and thank you for visiting our job post.


Supercoder is an AI-powered career development platform connecting developers worldwide to remote job opportunities with competitive payment.

  • Type of work: 100% Remote


Overview:

The client is hiring Python/Linux Engineers to design complex system-level evaluation tasks for LLMs. Design advanced benchmark tasks that evaluate the capabilities of modern Large Language Models (LLMs) such as ChatGPT, Claude, and other AI systems.

This role focuses on building realistic, technically challenging engineering scenarios that test model reasoning, debugging, and problem-solving abilities.

What You Will Do

  • Design complex, realistic engineering tasks to evaluate LLM reasoning, coding, debugging, and system understanding.
  • Build Python- and Linux-based workflows, pipelines, and multi-step scenarios.
  • Create reproducible environments using Python, Shell, and CLI tools.
  • Develop tasks that measure code comprehension, debugging, refactoring, and optimization.
  • Write clear technical documentation: problem statements, constraints, expected outputs, and detailed edge cases.
  • Use LLM tools (ChatGPT, Claude, etc.) to validate tasks and analyze model performance.

Must-Have Qualifications

  • 5+ years of professional software development experience.
  • Strong Python: modular code design, debugging complex programs, structured codebases.
  • Proficiency with Linux, Shell scripting, Bash, and command-line tools.
  • Solid technical English writing ability.
  • Strong reasoning, analytical thinking, and problem-solving skills.
  • Ability to design logical multi-step engineering scenarios.

Nice-to-Have Skills

  • Experience creating benchmark datasets, online judge problems, coding tests, or technical challenges.
  • Background with ICPC, Codeforces, Kaggle, or competitive programming.
  • Familiarity with Docker, Git, and CI/CD pipelines.
  • Experience with ML/AI or data-intensive engineering environments.

Who Will Excel in This Role

  • Engineers who enjoy designing difficult problems rather than simple feature development.
  • Developers who are strong at debugging, identifying subtle issues, and understanding complex system interactions.
  • Engineers who work well independently and can define their own approach.
  • Individuals interested in LLM evaluation, AI reliability, and technical task design.


Required Skill Profession

Computer Occupations



Your Complete Job Search Toolkit

✨ Smart • Intelligent • Private • Secure

Start Using Our Tools

Join thousands of professionals who've advanced their careers with our platform

Rate or Report This Job
If you feel this job is inaccurate or spam kindly report to us using below form.
Please Note: This is NOT a job application form.


    Unlock Your Python Systems Potential: Insight & Career Growth Guide


Advance your career or build your team with Expertini's smart job platform. Connecting professionals and employers in Vellore, India.