Finance Expert for AI Model Evaluation Project (Remote)
Ansprechpartner: InfogInnovative solutions Pvt Ltd
-, Indien
Endkundenprojekt
FinanzenBusiness AnalysisCorporate FinanceInvestment BankingProgram Evaluation And Review TechniqueSAP FI/COEquityEvaluierungsprojekteFintechArtificial Intelligence Markup Language (AIML)Beschreibung
Project Goal: Experienced Finance Professionals and Finance Subject Matter Experts (SMEs) are sought to support an AI Model Evaluation Project. The selected professionals will evaluate AI-generated responses in finance-related, multi-turn conversations by comparing responses against provided source documents and predefined evaluation guidelines. The project focuses on assessing accuracy, reasoning, contextual understanding, instruction-following, and reliability of AI-generated responses.
Main Tasks:
- Reviewing source documents, user conversations, and AI-generated responses
- Evaluating AI responses for factual and financial accuracy
- Reviewing multi-turn conversations while maintaining complete context
- Checking numerical calculations and financial reasoning
- Verifying whether AI responses correctly follow user instructions
- Identifying hallucinations, unsupported conclusions, and reasoning errors
- Determining whether each response is Correct or Incorrect
- Identifying and categorizing the primary reason for incorrect responses
- Selecting the appropriate failure category based on predefined guidelines
- Providing concise and technically accurate justifications for evaluation decisions
- Following project-specific SOPs and evaluation guidelines consistently
- Maintaining high accuracy, quality, consistency, and confidentiality
Required Skills:
- Strong knowledge of Finance, Accounting, Financial Reporting, or Business Analysis
- Excellent analytical, logical, and critical-thinking abilities
- Ability to accurately interpret financial and business information
- Strong English reading and written communication skills
- Excellent attention to detail
- Ability to review lengthy multi-turn conversations and maintain context
- Ability to identify factual, numerical, logical, and reasoning errors
- Ability to follow detailed instructions, guidelines, and SOPs
- Strong documentation and reporting skills
- Ability to work independently and meet project deadlines
Preferred Qualifications:
- CA (Chartered Accountant), CFA, CPA, MBA in Finance
- Master's or Bachelor's Degree in Finance, Accounting, Commerce, Economics, Financial Engineering, or Business Analytics
Preferred Professional Experience:
- Financial Analysis, Equity Research, Investment Banking, Financial Reporting, Accounting, Auditing, Credit Analysis, FP&A, Risk Analysis, Business Analysis, Investment Analysis
- AI Data Annotation, LLM Evaluation, AI Model Evaluation, Prompt Evaluation, Content Review, or Quality Assurance
Expected Results:
For each AI-generated response, the evaluator will verify factual correctness, check numerical calculations and financial logic, validate the response against the provided source document, check whether the AI correctly understood the conversation context, verify compliance with user instructions, detect hallucinations or unsupported claims, identify reasoning or logical errors, assign the appropriate failure category if applicable, and provide a concise technical justification for each decision.
Work Model: Remote
Main Tasks:
- Reviewing source documents, user conversations, and AI-generated responses
- Evaluating AI responses for factual and financial accuracy
- Reviewing multi-turn conversations while maintaining complete context
- Checking numerical calculations and financial reasoning
- Verifying whether AI responses correctly follow user instructions
- Identifying hallucinations, unsupported conclusions, and reasoning errors
- Determining whether each response is Correct or Incorrect
- Identifying and categorizing the primary reason for incorrect responses
- Selecting the appropriate failure category based on predefined guidelines
- Providing concise and technically accurate justifications for evaluation decisions
- Following project-specific SOPs and evaluation guidelines consistently
- Maintaining high accuracy, quality, consistency, and confidentiality
Required Skills:
- Strong knowledge of Finance, Accounting, Financial Reporting, or Business Analysis
- Excellent analytical, logical, and critical-thinking abilities
- Ability to accurately interpret financial and business information
- Strong English reading and written communication skills
- Excellent attention to detail
- Ability to review lengthy multi-turn conversations and maintain context
- Ability to identify factual, numerical, logical, and reasoning errors
- Ability to follow detailed instructions, guidelines, and SOPs
- Strong documentation and reporting skills
- Ability to work independently and meet project deadlines
Preferred Qualifications:
- CA (Chartered Accountant), CFA, CPA, MBA in Finance
- Master's or Bachelor's Degree in Finance, Accounting, Commerce, Economics, Financial Engineering, or Business Analytics
Preferred Professional Experience:
- Financial Analysis, Equity Research, Investment Banking, Financial Reporting, Accounting, Auditing, Credit Analysis, FP&A, Risk Analysis, Business Analysis, Investment Analysis
- AI Data Annotation, LLM Evaluation, AI Model Evaluation, Prompt Evaluation, Content Review, or Quality Assurance
Expected Results:
For each AI-generated response, the evaluator will verify factual correctness, check numerical calculations and financial logic, validate the response against the provided source document, check whether the AI correctly understood the conversation context, verify compliance with user instructions, detect hallucinations or unsupported claims, identify reasoning or logical errors, assign the appropriate failure category if applicable, and provide a concise technical justification for each decision.
Work Model: Remote