Now it’s possible to add vision support to your forms! Check out these examples and the source code below.

Vision by Image Upload

Vision by Image URL

_{Edited 2024-06-06 for compatibility with latest version of AI Engine.}

Installation

For only the file upload version, add the following JavaScript to a post or page with an AI Engine form on it:

jQuery(document).ready(function() {
    var input = document.querySelector('input[name="IMAGE"]');
    input.type = 'hidden'; // Make the existing input hidden

    // Create a new input for image upload
    var imageInput = document.createElement('input');
    imageInput.type = 'file';
    imageInput.accept = 'image/jpeg, image/png, image/webp'; // Accept only images

    // Insert the new image input after the hidden input
    input.parentNode.insertBefore(imageInput, input.nextSibling);

    // Function to handle image file selection and conversion to base64
    imageInput.addEventListener('change', function(event) {
        var file = event.target.files[0];
        if (file) {
            var reader = new FileReader();
            reader.onload = function(e) {
                var base64String = e.target.result;
                input.value = base64String; // Set the hidden input value to the base64 string
                var event = new Event('change', { bubbles: true });
                input.dispatchEvent(event);
            };
            reader.readAsDataURL(file); // Convert the image file to a base64 string
        }
    });
});

For both versions add the following PHP snippet:

function ai_convert_image_to_jpeg($input, $max = 4000) {
    // Check if input is a data URI
    if (strpos($input, 'data:image/') === 0) {
        // Extract MIME type and data from data URI
        list($type, $data) = explode(';', $input);
        list(, $data) = explode(',', $data);
        $data = base64_decode($data);
        $mime = str_replace('data:', '', $type);

        // Create an image resource from the data
        $image = imagecreatefromstring($data);
        if (!$image) return null; // Failed to create image

        // Determine MIME type for further processing
        $mime = (function_exists('getimagesizefromstring')) ? getimagesizefromstring($data)['mime'] : $mime;
    } else {
        // Fetch the image from URL
        $image_info = getimagesize($input);
        $mime = $image_info['mime'];

        switch ($mime) {
            case 'image/jpeg':
                $image = imagecreatefromjpeg($input);
                break;
            case 'image/png':
                $image = imagecreatefrompng($input);
                break;
            case 'image/webp':
                $image = imagecreatefromwebp($input);
                break;
            default:
                return null; // Unsupported format or failed to get image
        }
    }

    // Get original dimensions
    $orig_width = imagesx($image);
    $orig_height = imagesy($image);

    // Calculate new dimensions and resize if necessary
    if ($orig_width > $max || $orig_height > $max) {
        $ratio = $orig_width / $orig_height;
        if ($ratio > 1) {
            // Landscape
            $new_width = $max;
            $new_height = round($max / $ratio);
        } else {
            // Portrait
            $new_height = $max;
            $new_width = round($max * $ratio);
        }

        // Create new image and copy resized
        $new_image = imagecreatetruecolor($new_width, $new_height);
        imagecopyresampled($new_image, $image, 0, 0, 0, 0, $new_width, $new_height, $orig_width, $orig_height);
        imagedestroy($image); // Destroy original image resource
        $image = $new_image; // Update $image to point to the new resized image
    }

    // Process and convert the image to JPEG in memory
    ob_start();
    imagejpeg($image);
    $jpeg_data = ob_get_clean();

    // Cleanup
    imagedestroy($image);

    // Return the JPEG data
    return $jpeg_data;
}

add_filter( 'mwai_ai_query', function ( $query ) {
    if ($query->scope != 'form' || !isset($query->extraParams) || !isset($query->extraParams['fields']) || !isset($query->extraParams['fields']['IMAGE'])) { return $query; }
    $image_field = $query->extraParams['fields']['IMAGE'];
    $prefix = '['.$image_field.']';
    $query->message = mb_substr($query->message, strlen($prefix));
    if (str_contains($query->message,'data:image/')) {
        throw new Exception('There was an error with the image upload.');
    }
    $image_data = ai_convert_image_to_jpeg($image_field);
    if (!$image_data) { throw new Exception('There was an error with the image conversion.'); }
    $mimeType = 'image/jpeg';
    $query->set_file(new Meow_MWAI_Query_DroppedFile($image_data, 'data', 'vision', $mimeType));
    return $query;
}, 1, 1);

Optionally, edit $max = 1024 to the maximum image width and height you want to allow before it’s automatically resized.
https://docs.anthropic.com/claude/docs/vision#image-costs

Add Field to Form

Create an input field with the name IMAGE and a field type of Input.

Add [{IMAGE}] to the beginning of the prompt on the submit element. It will be removed before being sent to your connected AI model.

---

Let’s enhance your business! I can help you develop custom AI engine extensions or broader AI strategies.
Contact me.