Now it’s possible to add vision support to your forms! Check out these examples and the source code below.

Vision by Image Upload

Vision by Image URL

Installation

For only the file upload version, add the following JavaScript to a post or page with an AI Engine form on it:

jQuery(document).ready(function() {
    var input = document.querySelector('input[name="IMAGE"]');
    input.type = 'hidden'; // Make the existing input hidden

    // Create a new input for image upload
    var imageInput = document.createElement('input');
    imageInput.type = 'file';
    imageInput.accept = 'image/jpeg, image/png, image/webp'; // Accept only images

    // Insert the new image input after the hidden input
    input.parentNode.insertBefore(imageInput, input.nextSibling);

    // Function to handle image file selection and conversion to base64
    imageInput.addEventListener('change', function(event) {
        var file = event.target.files[0];
        if (file) {
            var reader = new FileReader();
            reader.onload = function(e) {
                var base64String = e.target.result;
                input.value = base64String; // Set the hidden input value to the base64 string
                var event = new Event('change', { bubbles: true });
                input.dispatchEvent(event);
            };
            reader.readAsDataURL(file); // Convert the image file to a base64 string
        }
    });
});

For both versions add the following PHP snippet:

function ai_convert_image_to_jpeg($input, $max = 4000) {
    // Check if input is a data URI
    if (strpos($input, 'data:image/') === 0) {
        // Extract MIME type and data from data URI
        list($type, $data) = explode(';', $input);
        list(, $data) = explode(',', $data);
        $data = base64_decode($data);
        $mime = str_replace('data:', '', $type);

        // Create an image resource from the data
        $image = imagecreatefromstring($data);
        if (!$image) return null; // Failed to create image

        // Determine MIME type for further processing
        $mime = (function_exists('getimagesizefromstring')) ? getimagesizefromstring($data)['mime'] : $mime;
    } else {
        // Fetch the image from URL
        $image_info = getimagesize($input);
        $mime = $image_info['mime'];

        switch ($mime) {
            case 'image/jpeg':
                $image = imagecreatefromjpeg($input);
                break;
            case 'image/png':
                $image = imagecreatefrompng($input);
                break;
            case 'image/webp':
                $image = imagecreatefromwebp($input);
                break;
            default:
                return null; // Unsupported format or failed to get image
        }
    }

    // Get original dimensions
    $orig_width = imagesx($image);
    $orig_height = imagesy($image);

    // Calculate new dimensions and resize if necessary
    if ($orig_width > $max || $orig_height > $max) {
        $ratio = $orig_width / $orig_height;
        if ($ratio > 1) {
            // Landscape
            $new_width = $max;
            $new_height = round($max / $ratio);
        } else {
            // Portrait
            $new_height = $max;
            $new_width = round($max * $ratio);
        }

        // Create new image and copy resized
        $new_image = imagecreatetruecolor($new_width, $new_height);
        imagecopyresampled($new_image, $image, 0, 0, 0, 0, $new_width, $new_height, $orig_width, $orig_height);
        imagedestroy($image); // Destroy original image resource
        $image = $new_image; // Update $image to point to the new resized image
    }

    // Process and convert the image to JPEG in memory
    ob_start();
    imagejpeg($image);
    $jpeg_data = ob_get_clean();

    // Cleanup
    imagedestroy($image);

    // Return the JPEG data
    return $jpeg_data;
}

add_filter( 'mwai_ai_query', function ( $query ) {
    if ($query->scope != 'form' || !isset($query->extraParams) || !isset($query->extraParams['fields']) || !isset($query->extraParams['fields']['IMAGE'])) { return $query; }
    $image_field = $query->extraParams['fields']['IMAGE'];
    $prefix = '['.$image_field.']';
    $query->message = mb_substr($query->message, strlen($prefix));
    if (str_contains($query->message,'data:image/')) {
        throw new Exception('There was an error with the image upload.');
    }
    $image_data = ai_convert_image_to_jpeg($image_field);
    if (!$image_data) { throw new Exception('There was an error with the image conversion.'); }
    $mimeType = 'image/jpeg';
    $query->set_file(new Meow_MWAI_Query_DroppedFile($image_data, 'data', 'vision', $mimeType));
    return $query;
}, 1, 1);

Optionally, edit $max = 1024 to the maximum image width and height you want to allow before it’s automatically resized.
https://docs.anthropic.com/claude/docs/vision#image-costs

Add Field to Form

Create an input field with the name IMAGE and a field type of Input.

Add [{IMAGE}] to the beginning of the prompt on the submit element. It will be removed before being sent to your connected AI model.

Let’s enhance your business! I can help you develop custom AI engine extensions or broader AI strategies.
Contact me.