About this opportunity
AI Operations Specialist Workflow Automation & Quality - Remote, USA | Temporary, Full-Time | Rose International Menu 1 Menu 2 Menu 3 Menu 4 --> Rose Home Rose News Contact Rose AI OPERATIONS SPECIALIST WORKFLOW AUTOMATION & QUALITY JOB IN Remote, USA! Thank you for your interest in working with Rose International! Unfortunately, the job you are trying to apply for is no longer available. Please visit our Jobs Page to find a new opportunity. New positions are posted daily! Thank you for your interest in working with Rose International! Unfortunately, the job you are trying to apply for has reached its maximum number of applicants. Please visit our Jobs Page to find a new opportunity. New positions are posted daily! Benefits: For information and details on employment benefits offered with this position, please visit here . Should you have any questions/concerns, please contact our HR Department via our secure website . California Pay Equity: For information and details on pay equity laws in California, please visit the State of California Department of Industrial Relations' website here . Rose International is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, age, sex, sexual orientation, gender (expression or identity), national origin, arrest and conviction records, disability, veteran status or any other characteristic protected by law. Positions located in San Francisco and Los Angeles, California will be administered in accordance with their respective Fair Chance Ordinances. If you need assistance in completing this application, or during any phase of the application, interview, hiring, or employment process, whether due to a disability or otherwise, please contact our HR Department . Rose International has an official agreement (ID #132522), effective June 30, 2008, with the U.S. Department of Homeland Security, U.S. Citizenship and Immigration Services, Employment Verification Program (E-Verify). (Posting required by OCGA 13/10-91.). About Rose Founded in 1993 Office Locations Across the U.S. 150+ Clients: Corporations and Government Agencies Employee Oriented Company Challenging Assignments Across the U.S. Continuous Professional Development Thanks for the opportunity. If in the future I ever need a job, I would like to work for Rose International. David, Consultant I had a very positive experience working for Rose. The entire process is very efficient and easy. Joanne, Consultant It is a great pleasure being a part of the Rose International Team. Toni, Consultant Rose International has been great to me. I thank everyone there for all of their hard work; it has not gone unnoticed. Melody, Consultant My on-boarding with Rose was outstanding. The packets of information, the process, and great attention to detail each person gave me allowed me to get started quickly.I appreciated each person's friendly and helpful attitude. Diana, Consultant EMPLOYEE COMMENTS We want you to work with us, but don't take our word for it. Take a look at this sampling of employee comments. They speak for themselves. AI Operations Specialist Workflow Automation & Quality Required Skills:• 5-7 years in operations, program management, analytics, consulting or applied AI, where you were responsible for results.• You use AI every day and have built something with it that has actually moved the needle, such as a workflow, an automation or an analysis. You know where it breaks, and you keep pushing it further.• You're comfortable with numbers: accuracy, precision and recall, sampling. SQL and notebooks help. If you haven't used them yet, you'll pick them up quickly with AI assistance.• You're curious about how work actually gets done, including the awkward exceptions, and you've taken a messy process and made it measurable.• You've worked on quality in some form, such as review guidelines, QA or audits.• You write clearly and can work with business leads, PMs and engineers in the same week.• You're fine working with limited direction, and you try things yourself instead of waiting for someone to build them. A technical background isn't required.• You don't need to have designed evaluations or calibrated LLM judges before. We'll teach you, and you'll work with a team that has done it many times.Preferred experience• Hands-on SQL, Python or API experience.• Trust and safety, content moderation, annotation operations or other human-in-the-loop work at scale.• Fine-tuning models or calibrating LLM judges.• Background in a two-sided marketplace, travel, or another operations-heavy business.• Experience setting up intake and prioritization for a new team.Duties:We're the AI & ML Enablement team in Supply Tech. We help teams across Client use AI to do real work, and prove that it works. Right now, we're helping business and field teams use AI to take on the review and quality work they run, and helping product teams understand whether an AI feature works before it reaches our users. Day to day, we move manual review processes to AI and put people where their judgment matters most. We design new workflows so AI does the work from the start. We also build the labeled datasets that show whether the AI is right, and we keep everything running as models and policies change. Our methods and projects change as the models do, and we expect the scope of this role to grow with them.The difference you will make:You'd work alongside our team on whichever problems AI could take on, whether that's a business team with a process that's slow, expensive or inconsistent, or a product team getting an AI feature ready to launch. The job looks much the same either way. You start with how the work really gets done, help build an AI version of it, and test it until you trust the results. Then you hand it off and use what worked on the next problem. The mix of projects will keep changing as AI takes on more of the work, so we care more about how you work than about a fixed list of projects. This is a hands-on role. You help build the workflow, ship it, and stay on to make sure it works.A typical day:• Help the team decide which projects will drive the largest business impact, and make the case for the ones worth doing.• Sit with the team that owns a problem, map how the work gets done today, and work out how AI can do it and where people step in for the hard cases.• Build the AI workflow: write the prompts, set up the pipeline, and keep it at the agreed quality level, watching for drift once it's live.• Build the test sets, measure accuracy, and calibrate LLM judges against human reviewers.• Support product teams in agreeing on what "good enough to launch" means for an AI feature, and bring the evidence for the go/no-go call.• Turn unclear policy into criteria a reviewer can apply consistently, working with product, Legal and Policy.• Work with subject matter experts to write labeling guidelines and produce human-verified labels on real data.• Run labeling at scale with vendor agents, and check their work against agreement metrics and your own spot checks.• Make what you build reusable and document what you learn, so the next team gets to production faster.• Keep our methods current as models, tools and vendor agents change, and retire what stops working.• Own the workstreams you're assigned, from setup through quality checks. Model infrastructure and product roadmaps stay with the teams that own them.**Only those lawfully authorized to work in the designated country associated with the position will be considered.** **Please note that all Position start dates and duration are estimates and may be reduced or lengthened based upon a client’s business needs and requirements.**
What the employer listed
- Pay
- Salary Not Listed
- Schedule
- Not listed
- Experience
- Not listed
- Education
- Not listed
- Benefits
- Not listed
- Eligible locations
- Remote — USA