AI-integrated tool for creative professionals
A collaborative web app for generating video storyboards, texts, and pictures
*We developed this project under an NDA, so we can’t disclose its name or provide a link to the product.
Project idea
A creative agency owner from Saudi Arabia approached us to develop an application that helps professionals like screenwriters, designers, and copywriters save time on creating marketing materials.
The aim is to let them focus more on the creative aspects of their work, rather than manual tasks such as drawing, writing, and compiling video storyboards.
Client’s goal
Our client recognized that generative AI models are well-suited for creative tasks and sought to develop an application for creative professionals that takes advantage of this technology. While the client's technical team could develop the app, they lacked expertise in AI integration and were uncertain about which agency processes could benefit from the technology. Consequently, they asked us to guide them in identifying how AI could improve their business processes and to design a simple, user-friendly interface for the creative web tool.
Action plan
Do analytical research to find out how the client’s business works and find the most time-consuming procedures that could benefit from AI integration.
Design the web app’s interface and make it intuitive for the client’s employees and simple to develop.
Analytics
In the very beginning, the client approached us with a long file with the description of the application he wanted to develop. It displayed about 7 blocks with numerous features that could be applied to different processes within the client’s creative agency.
The development of such an application would have been too long and expensive; that’s why, together with the client, we decided to conduct analytics research to determine which features are the most needed and would benefit from AI integration.
Video production is a major process within the client’s agency. To determine where AI integration could be beneficial, we analyzed the standard video production cycle followed by the client’s team. We broke down the workflow into individual steps to understand the specific tasks employees perform at each stage.
Gathering requirements
The team begins by understanding the client's vision and ideas for the marketing campaign, conducting market research, and developing a strategic approach for the campaign.
Based on the client-provided brief, the team develops creative concepts. If the client is not satisfied, the team proposes alternative options. Once finalized, the team presents the materials to the client.
Preparing for production
The team completes work on materials such as storyboards and scripts and schedules production sessions.
Production
The team proceeds with shooting, editing, and processing the video material to make sure it's ready for distribution across marketing channels.
Creating content and gathering feedback
The team produces the necessary texts and images for the campaign, making sure all required content is finalized, and collects feedback from the client.
This graph revealed that the most resource-intensive task in the agency's video production is revising storyboards when clients request changes to the scenario. The team currently redraws scene images and types new text manually. Implementing an AI model could automate these tasks, significantly saving the creative team time and effort.
As a result of such an analysis, together with the client, we have decided that AI would be used to create pictures and texts for storyboards.
Figuring out how AI can help with storyboard generation
We moved on to understand how exactly the application would work and started experimenting with AI models such as GPT4o and DALL-E 3. We tried to generate video frames and texts and analysed the result to better understand the limitations of the model. At first, we generated the entire scenario as a single picture.
Although such a method helped to achieve great consistency, it didn’t give the storyboard creator necessary control over important details. It also didn’t allow the user to change different parts of the scene one by one. Therefore, we decided that each frame of the storyboard should be generated independently.
To achieve consistency of the scenario and characters within the frame, we introduced a character editor where a user can describe the character of the video in detail. We have decided that the editor is necessary since it could provide the storyboard creator with greater control over the little details of each picture.
This feature increases the likelihood that the AI model generates images that align precisely with the creator's vision. These experiments showed how AI can assist in storyboard creation as well as help us and the client identify the effective way to organize the process.
Designing in-app architecture
After we experimented with AI, we began planning how the blocks and sections would be organized in the app. Our analysts created a scheme that illustrates the interconnections between different entities of the application and how the app will function.
We discussed and approved the scheme with the client to ensure we have a shared vision of the AI-integrated app. This scheme is essential for the design stage, as it anchors the in-app architecture that will be reflected in the design and prevents the need for extensive redesigns that consume significant time and resources.
This scheme helped both us and our client understand how the AI-integrated app would function from a technical perspective.
Planning for the growth
Since our client planned to expand the AI-integrated app for broader functionality and opportunities, we created a project roadmap. This outlined which features would be included in the app's initial version and which would be added in later stages.
It provided both us and our client with a clear definition of key milestones and deliverables, enabling us to track progress against our objectives.
How the app works
Creating a campaign
At the beginning, the user can create a campaign and provide information about the brand, its voice, the campaign objectives, the target audience, the value proposition, and other details. These details will later be incorporated into the model, giving it more information to generate content.
Creating a new storyboard
To create a new storyboard that will serve as the backbone of the video reel, a user can either start typing the scenario or generate it using an AI prompt.
Once the initial scenario description is completed, the app automatically identifies the video characters mentioned in the text and organizes them into separate character profiles.
Editing a storyboard
After the screenplay description is completed and character profiles are filled, users can employ AI to generate a storyboard. This storyboard breaks down the entire video scenario into scenes, each accompanied by an AI-generated picture depicting characters and their surroundings relevant to the video.
Completing character profiles
Names and descriptions of characters in videos can be extracted from the text the user inputs or created manually. Each character has a profile containing details such as age, gender, and ethnicity, which are later integrated into the AI model to generate consistent video frames featuring these characters.
A detailed description is essential to ensure that the AI accurately represents the characters’ appearances without adding any unnecessary details.
Scene descriptions can be edited manually or with AI assistance. Pictures can also be modified using predefined or custom prompts.
Users have the option to change images individually or regenerate an entire scene, with the ability to revert to previous versions if the result is unsatisfactory.
Shot details
Once the images are generated, the storyboard creator can add descriptions. They specify details such as frame size, camera movement, sound effects, voiceover scripts, frame duration, and other vital elements. This functionality is valuable for thoroughly planning the video scenario and outlining all necessary resources.
Export to PDF
Upon completion, the storyboard can be exported as a PDF and shared with clients or operators on any medium, including paper. This feature provides maximum convenience, allowing the storyboard to be used without accessing the application.
Using collaborative features
This AI-integrated app is designed for collaborative work, allowing users to leave comments and conduct material reviews directly within the app. Additionally, projects can be shared among users and are stored in a central location, similar to Google Drive.
Version history
At any stage of the process, the user can view and revert to any previous version of the project, making the application more flexible. The app features autosave, which runs every 10 minutes and saves all stages of the project. Users can also access the version history and manually save the current version by clicking the "Create snapshot" button.
Each version is labeled as either "Autosave" or "Manual snapshot", allowing the user to easily distinguish between automatically and manually saved versions and navigate the version history more quickly.
Conducting presentations
After finalizing the storyboards and other materials, the agency's workflow culminates in presenting them to the customer. To streamline this process, we incorporated features that automatically compile these materials into a cohesive presentation. This presentation includes content in various formats, such as social media posts of different sizes and videos and images displayed in mobile and TV screen mockups.
Developing content plans
The content plan consists of a list of publications for various social networks and supports collaborative work. Employees responsible for creating illustrations can contribute these to the cards, while other users can add additional materials needed for publication, such as music and animations.
Once complete, all creative materials can be handed over to managers in charge of publishing content. While this feature does not involve AI, it addresses a resource-intensive process within the client's agency.
UI/UX design
When creating UI/UX design for this AI-integrated app, we relied on the principles and schemas established during the analytics phase. Our focus was on developing intuitive navigation and features rather than visual style, as well as on making the application easy to develop when it is handed over to the client’s development team.
Design concept
We began by creating a design concept, which included a few screens showcasing the color scheme, typography, navigation patterns, and layout of the key functional elements of the future app. Once the design concept was finalized, we discussed it with the client and then proceeded to design the full interface.
Consistency
In this AI-integrated app, AI is used to generate various entities, including pictures and texts. We endeavored to ensure that the functionality for entering prompts operates consistently throughout the app to make AI features transparent to creative professionals and minimize confusion.
Additionally, we standardized other functionalities across the entire interface. For example, features for regenerating images, reverting to previous versions, changing picture styles, and adjusting storyboard settings remain consistent at every point of interaction with the storyboard and the application.
Familiar navigation patterns
In this AI-integrated app, we positioned the navigation bar on the left side, allowing users to move between different panels and storyboards generated by AI. This design pattern is similar to ChatGPT's interface.
Additionally, we organized comments on the right side, following a layout similar to Figma. Since these patterns are familiar to many creative professionals, the interface feels intuitive and user-friendly to them.
Applying our company’s design system
Our team has completed numerous design projects. Over time, we have developed a comprehensive design system comprising ready-made UI elements that can be reused in new projects. These elements ensure consistency in design and allow us to focus less on manual UI tasks and more on creating principles that make the design convenient and user-centric. Additionally, the design system helps us speed up the design process.
Gathering a UI kit
We assembled a UI kit containing the elements used during the design process. These ready-made components can be used by our team or the client to scale the design in the future. Furthermore, the UI elements in the kit are directly connected to the code, enabling the client’s team to focus less on layout development and more on backend setup and AI integration.
Results
Our team completed the entire mockup of the AI-integrated app within four months and delivered it to the client’s development team. The client was satisfied with the design, leaving no negative feedback, and only requested us to add more functionality to the app. Further tasks, such as AI integration and coding, are now the client's responsibility.
One of the key factors contributing to our success with this project was the analytics research. This research enabled us to identify business processes within the client’s company that could be enhanced with AI and to understand precisely how AI could be integrated into the interface of such a creative tool. Analytics also helped us refine the list of potential app features, making the design and development stages less time-consuming and costly for our client.
In our opinion, in-depth analytics is an essential step in planning AI integration, as it helps determine the value AI can bring to each business and how it can be technically implemented in each specific case.
