Job Title: Senior QA Engineer – Web, Mobile & Conversational AI Testing
Location: Remote
Experience Level: 4+ years
Employment Type: Contract
About the Role:
We are looking for a highly experienced Senior QA Engineer to lead and execute quality assurance efforts across our web, mobile, and AI-driven conversational products. You will play a critical role in ensuring our applications deliver exceptional user experiences, high performance, and reliability—especially in LLM-based (Large Language Model) conversational systems.
Key Responsibilities:
Test Planning & Strategy
Develop and maintain comprehensive test plans and test cases for web, mobile, and conversational AI applications.
Define QA strategy for end-to-end testing of features including functional, UI/UX, performance, and regression testing.
Execution & Automation
Perform manual and automated testing for mobile (iOS/Android) and web platforms. [Selenium, Pytest, Python Scripts]
Design and maintain automated test scripts using tools like Selenium, Appium, or Playwright.
Conduct conversational flow and response accuracy testing for LLM-based chat interfaces.
LLM & Conversational App Testing
Evaluate dialogue quality, prompt-response relevance, tone, and fallback behaviors in GPT-like systems.
Test for hallucinations, safety, toxicity, and coherence in AI outputs.
Validate integrations with RAG systems, vector stores, and API workflows.
Cross-Functional Collaboration
Collaborate with product managers, developers, and AI engineers to ensure feature clarity and testability.
Provide feedback early in the development lifecycle to reduce bugs and enhance quality.
Monitoring & Reporting
Track and report bugs using tools like JIRA or Linear.
Analyze and report on test results, coverage, and quality trends.
Required Qualifications:
4+ years of experience in QA for web and mobile applications.
Hands-on experience with both manual and automation testing tools and frameworks.
Strong knowledge of QA methodologies, testing types, and software development lifecycle.
Automation testing using Pytests.
Experience testing LLM-based or NLP-powered applications (ChatGPT, Rasa or custom LLMs).
Proficient in writing SQL queries and working with Postman for API testing.
Familiarity with vector databases and conversational testing tools (Langfuse, Phoenix, etc.) is a plus.
Preferred Skills:
Experience in testing AI-driven personalization, intent recognition, or multi-turn conversation logic.
Exposure to prompt engineering and evaluation frameworks for LLM outputs.
Familiarity with CI/CD pipelines and version control systems like Git.
Ability to build QA dashboards and automated quality reports.