Gemini AI API with LWC : File Analysis

sanketthoughts

12 months ago

🧠 File Analysis using Gemini-2.5-flash REST API

(Agentforce Models API not required, The API is called directly through the JS, For better security control, Please try to route the call through a proxy server e.g. node js with render or heroku)

You can check a working example here about using Agentforce Models API: Agentforce Models API & LWC: File Analysis

⚠️ Caution: While the Agentforce Models API offers built-in security and governance, using a direct public LLM endpoint like Gemini-2.5-flash introduces potential security implications. If you choose this approach, be sure to implement your own guardrails, input validation, and data protection measures.

In this tutorial, we’ll explore how to analyze file using public LLMs (like Gemini) via Lightning Web Components (LWC). This method sends user uploaded files to an external LLM through input prompt and returns AI-generated analysis.

This approach allows you to:

Select any file locally stored in your system
Send the file along with user instructions or prompts to Gemini
Return the AI-generated analysis to your Salesforce UI

🛠 LWC: JsBasedGeminiApiCall

JsBasedGeminiApiCall.html

File Upload Input: Allows users to upload a video file.
Prompt Text Area: Lets users enter a text prompt.
Send Button: Sends the uploaded file and prompt to the Gemini API when clicked (button is disabled during loading).
Loading Spinner: Displays a spinner while waiting for the API response.
Response Display: Shows the API’s response once received.
File Name Display: Shows the name of the uploaded file after selection.

<template>
    <lightning-card title="Gemini API Tester" icon-name="custom:custom63">
        <div class="slds-m-around_medium">
            <lightning-input type="file" label="Upload Video" onchange={handleFileChange}></lightning-input>

            <!-- Show uploaded file name -->
            <template if:true={fileName}>
                <p class="slds-text-body_small slds-m-top_xx-small">File: {fileName}</p>
            </template>

            <lightning-textarea label="Text Prompt" onchange={handlePromptChange}></lightning-textarea>

            <lightning-button label="Send to Gemini" onclick={callGeminiAPI}
                class="slds-m-top_medium" disabled={isLoading}></lightning-button>

            <!-- Spinner while loading -->
            <template if:true={isLoading}>
                <div class="slds-m-top_medium slds-align_absolute-center">
                    <lightning-spinner alternative-text="Analyzing..." size="medium"></lightning-spinner>
                </div>
            </template>
        </div>

        <!-- Show response -->
        <template if:true={response}>
            <div class="slds-m-around_medium">
                <h3 class="slds-text-heading_small">Response:</h3>
                <pre>{response}</pre>
            </div>
        </template>
    </lightning-card>
</template>

jsBasedGeminiApiCall.js

File Handling: Converts an uploaded video to Base64 using FileReader.
Prompt Input: Captures user text input for AI processing.
API Request: Sends a structured POST request to Gemini’s streamGenerateContent endpoint.
Response Handling: Extracts and displays the AI-generated text from the streamed response.
Error & Loading States: Manages UI loading spinner and displays error messages if needed.
⚠️ Note: The API key is hardcoded and should never be exposed in production. Use server-side proxies or secure storage instead.

import { LightningElement } from 'lwc';

export default class JsBasedGeminiApiCall extends LightningElement {
    base64VideoData;
    textPrompt = '';
    response = '';
    fileName = '';
    isLoading = false;

    handleFileChange(event) {
        const file = event.target.files[0];
        if (file) {
            this.fileName = file.name;
            const reader = new FileReader();
            reader.onloadend = () => {
                const base64 = reader.result.split(',')[1];
                this.base64VideoData = base64;
            };
            reader.readAsDataURL(file);
        }
    }

    handlePromptChange(event) {
        this.textPrompt = event.target.value;
    }

    async callGeminiAPI() {
        this.response = '';
        this.isLoading = true;

        const apiKey = 'Generate_API_Key_And_replace_here'; // ⚠️ Do NOT expose in production
        const endpoint = `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:streamGenerateContent?key=${apiKey}`;

        if (!this.base64VideoData || !this.textPrompt) {
            this.response = 'Video and prompt are both required.';
            this.isLoading = false;
            return;
        }

        const requestBody = {
            contents: [
                {
                    role: 'user',
                    parts: [
                        {
                            inlineData: {
                                mimeType: 'video/x-matroska',
                                data: this.base64VideoData
                            }
                        }
                    ]
                },
                {
                    role: 'user',
                    parts: [
                        {
                            text: this.textPrompt
                        }
                    ]
                }
            ],
            generationConfig: {
                thinkingConfig: {
                    thinkingBudget: -1
                },
                responseMimeType: 'text/plain'
            }
        };

        try {
            const res = await fetch(endpoint, {
                method: 'POST',
                headers: {
                    'Content-Type': 'application/json'
                },
                body: JSON.stringify(requestBody)
            });

            if (!res.ok) {
                throw new Error(`Gemini API error: ${res.statusText}`);
            }

            const result = await res.json();

            let readableText = '';
            result.forEach(entry => {
                const parts = entry?.candidates?.[0]?.content?.parts || [];
                parts.forEach(part => {
                    readableText += part.text + '\n';
                });
            });

            this.response = readableText.trim();
        } catch (err) {
            console.error(err);
            this.response = 'Error calling Gemini API: ' + err.message;
        } finally {
            this.isLoading = false;
        }
    }
}

jsBasedGeminiApiCall.js-meta.xml

<?xml version="1.0" encoding="UTF-8"?>
<LightningComponentBundle xmlns="http://soap.sforce.com/2006/04/metadata">
    <apiVersion>64.0</apiVersion>
    <isExposed>true</isExposed>
    <targets>
        <target>lightning__AppPage</target>
        <target>lightning__RecordPage</target>
        <target>lightning__HomePage</target>
    </targets>
</LightningComponentBundle>

🔍 Use Cases

Video interview analysis (e.g. candidate screening)
Meeting summary generation
Product demo review
Compliance audit video review
Explainer video feedback

✅ Summary & Demo

This architecture allows Salesforce apps to tap into powerful external LLMs like Gemini without requiring the Agentforce platform. While flexible and fast to implement, it does require extra responsibility when handling sensitive content or real-time data.

If you’re working in regulated industries or handling customer data, consider using Agentforce Models API or implementing enterprise-grade security controls before going live.

Do you need help?

Are you stuck while working on this requirement? Do you want to get review of your solution? Now, you can book dedicated 1:1 with me on Lightning Web Component and Agentforce completely free.

GET IN TOUCH