AI and the future of design: Designing with AI - Noah L, Jordan S, Andrew P, Vincent van der Meulen

23 Jun 202341:49

TLDRAt Config 2023, Noah Levin, Jordan Singer, Andrew Paranavithana, and Vincent van der Meulen from Figma discussed the integration of AI into design. They explored AI's potential to redefine design processes and tools, enabling more dynamic task-based computing and simplifying complex interfaces. The team shared insights on AI's role in enhancing brainstorming, design, and collaboration, emphasizing its supportive function in the creative process. The acquisition of Diagram, a company specializing in AI-driven design tools, was highlighted, showcasing the future of AI in enhancing design productivity and user experience.


  • 🌟 Config 2023 is a significant event for the Figma community, with a focus on AI and its integration into design processes.
  • 🤖 The keynote speaker, Noah Levin, VP of Design at Figma, highlighted AI's potential to transform design by focusing on problem-solving for people, rather than just pushing pixels.
  • 🔧 AI is seen as an evolution of design tools, not a replacement for thoughtful design. It's another step in the historical progression of design alongside technological advancements.
  • 🛠 The discussion emphasized three key areas of change through AI: what we design, how we design, and who designs, suggesting a shift towards more dynamic, task-based computing.
  • 💡 AI could simplify complex interfaces, focusing more on core tasks and allowing designers to spend less time on monotonous work and more on strategic decisions.
  • 🛑 The importance of maintaining the human element in design was stressed, with AI serving as a tool to assist and augment the design process rather than replace designers.
  • 🔄 The acquisition of Diagram by Figma was announced, indicating a strategic move to integrate AI more deeply into Figma's platform and enhance the design capabilities of its users.
  • 🌐 The potential of AI to democratize design by lowering the barrier to entry for non-designers and enabling more people to participate visually in the design process was discussed.
  • 📈 The integration of AI into the design process was presented as an opportunity to raise the ceiling of creative outputs and improve the quality of products.
  • 🤝 The Figma team encouraged community involvement, emphasizing the importance of feedback from designers to shape the future development of AI in their platform.
  • 🔮 The presentation by the Diagram team showcased the potential of AI to assist in brainstorming, design, building, and collaboration, positioning AI as a supportive tool throughout the entire design process.

Q & A

  • What is the main topic of the talk at Config 2023?

    -The main topic of the talk is AI and its implications for the future of design, specifically discussing how AI might change what we design, how we design, and who gets to design using tools like Figma.

  • What is the role of Noah L in the presentation?

    -Noah L is the VP of Design at Figma, and he provides insights into how AI might impact the design industry and how Figma is thinking about integrating AI into its platform.

  • What changes were made based on attendee feedback from the previous day's talks?

    -The organizers moved product deep dives into the main stage room and opened up the third floor for overflow to accommodate more attendees for talks that might be full on the second floor.

  • What is the significance of the acquisition of Diagram by Figma?

    -The acquisition of Diagram by Figma signifies Figma's commitment to integrating AI into its platform, as Diagram has been working on AI-driven design tools and has shown promising results in this area.

  • How does Noah L define design?

    -Noah L defines design as solving problems for people, emphasizing that it's not just about pushing pixels but also involves what designers do with their minds.

  • What are the two grounding assumptions Noah L starts with in his talk?

    -The two grounding assumptions are that design is an everlasting problem-solving process for people and that design has always evolved with technology.

  • What is the potential shift in the software we build due to AI, as mentioned by Noah L?

    -Noah L suggests a potential shift from an app-centric world to more dynamic, task-based computing, where AI might simplify complex interfaces and focus more on the core jobs to be done.

  • What is the concept of 'Magician' introduced by Andrew P?

    -Magician is a tool bag of different utilities that can be used inside of Figma, which leverages AI to assist with various design tasks without the need for writing code.

  • What is the 'Genius' project that Vincent van der Meulen discusses?

    -Genius is a project that aims to create a design assistant using AI, which can understand the design world and make suggestions on how to finish a design, all within the Figma platform.

  • What is the 'Genius Chat' feature introduced by Andrew P?

    -Genius Chat is an early prototype that allows users to have a direct conversation with the AI assistant, Genius, to make changes to their Figma files through text commands and receive feedback.



🎤 Conference Opening and Introduction to AI Discussion

Jenny Wen, a design manager at Figma, opens the conference by acknowledging the attendees' enthusiasm and the importance of the event. She addresses the previous day's issues with session attendance and announces changes to accommodate attendees better. The main focus of the day is AI, a topic of significant interest to the audience, and Noah Levin, VP of Design at Figma, is introduced to delve deeper into AI's role in the Figma platform and its potential impact on the design industry.


🤖 The Role of AI in Design and Its Evolution

Noah Levin discusses the fundamental aspects of design and its evolution with technology. He emphasizes that AI is not about replacing human designers but augmenting their capabilities. He uses historical examples to illustrate how design has adapted to technological advancements, such as the printing press and digital photography, and suggests that AI will be another such evolution, requiring designers to adapt and learn new ways of working.


🛠️ The Impact of AI on Design Processes and Tools

The speaker explores how AI might change what we design, how we design, and who designs. AI could simplify complex interfaces and shift from app-centric computing to more dynamic, task-based interactions. It could also transform the design process from manipulating individual elements to considering larger patterns, potentially freeing designers to focus more on strategy and taste rather than minutiae. The importance of design systems in this transition is highlighted.


🔍 AI's Influence on Design Collaboration and Accessibility

The paragraph discusses the potential shift in design authorship from specialized stakeholders to a more inclusive design partnership, where AI could lower the barrier for non-designers to participate visually. It also suggests that AI could raise the ceiling for professional designers, allowing for more ambitious creations. The importance of embracing collaboration and focusing on shared goals is emphasized.


🌟 Figma's Approach to Integrating AI

Figma's community-centered approach to AI integration is outlined, highlighting the company's desire to build tools that serve the needs of its users. The speaker discusses the early days of AI plugins and the potential for AI to assist in various stages of the design process, from brainstorming to building. The focus is on AI as a supportive tool rather than a replacement for human creativity.


🛑 Reflections on AI's Practical Applications in Design

The speaker reflects on the practical applications of AI in design, discussing the potential for AI to assist in brainstorming, designing, building, and collaborative processes. Examples of AI-generated code, design pattern suggestions, and prototyping enhancements are given. The emphasis is on AI's ability to streamline tasks and enhance the design workflow.


🔗 The Future of Design Tools with AI Integration

The paragraph outlines the journey of the Diagram team and their exploration of AI in design tools. It discusses the evolution of design tools and the integration of AI to enhance the design process. The team's experiments with AI, such as the automator plugin and design OS, are highlighted, showcasing the potential for AI to transform design workflows.


🚀 The Birth and Evolution of Diagram's Design Assistant

The paragraph details the birth of Diagram and its evolution towards creating a design assistant. It discusses the team's exploration of different products and the challenges they faced in representing visual designs to a language model. The introduction of 'Genius', a project that aimed to suggest design elements, is highlighted as a significant step in their journey.


💡 The Challenges and Iterations of Design AI Development

The speaker discusses the challenges faced in developing a reliable design AI, including the need for speed, accuracy, and effective communication. Various concepts, such as auto-suggestions and text-based design prompts, are explored, each with their own advantages and limitations. The goal is to create a tool that enhances the design process without replacing the designer's creativity.

🤝 The Integration of AI with Human Design Workflows

The paragraph explores the potential for AI to collaborate with human designers in real-time, similar to multiplayer collaboration in Figma. It discusses the idea of 'Genius Chat', which allows for direct interaction with the AI to make design suggestions and edits within the Figma canvas. The emphasis is on creating a tool that complements the designer's workflow and enhances their capabilities.

🌈 The Future of Design with AI and Human Collaboration

The final paragraph wraps up the discussion by emphasizing the continued importance of human designers and the exciting possibilities that AI integration offers. The speaker expresses excitement about the collaboration between Figma and the Diagram team and looks forward to exploring the future of design with AI, while inviting feedback from the community to shape this new intersection of technology and creativity.




AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions. In the context of the video, AI is central to the future of design, with its potential to transform how designers work by automating tasks, generating ideas, and enhancing the design process. The script discusses how AI can be integrated into platforms like Figma to assist in brainstorming, designing, and building phases of the design workflow.


Design, as mentioned in the video, is fundamentally about solving problems for people. It goes beyond the physical act of creating and involves a mental process as well. The video emphasizes that design has always evolved with technology, and AI represents the next step in this evolution, where it can assist designers in creating more effective and efficient solutions.


Figma is a cloud-based interface design and collaboration tool that is highlighted in the video as a platform that is exploring the integration of AI. The speakers discuss how Figma is thinking about bringing AI to its platform, which would allow for more dynamic and task-based computing, potentially changing the way designers interact with design software.


Config, as referenced in the script, is an event where the video's presentation takes place. It seems to be a conference or gathering focused on design and technology, where attendees learn, connect, and engage with the latest tools and ideas in the industry.


Diagram is a company that has been acquired by Figma, as announced in the script. Diagram has been working on integrating AI into design tools, and their work is seen as an exciting addition to Figma's capabilities. The team from Diagram shares their journey and insights on building a design assistant with AI during the presentation.

💡Design Systems

Design Systems are a set of reusable design components that ensure consistency and efficiency in product design. In the video, Design Systems are mentioned as a preparation for the integration of AI, as they provide a structured and pattern-based approach that AI can leverage to automate design tasks and suggest design elements.

💡Generative AI

Generative AI refers to AI models that can create new content, such as images, text, or designs, rather than just recognizing or analyzing existing content. The script discusses how generative AI can be used to generate UI elements or even entire design layouts, streamlining the design process.


Prototyping in the design context involves creating a preliminary model of a product to test and refine its features. The video mentions the use of AI in prototyping to create high-fidelity prototypes and even suggests the potential for AI to automate certain aspects of the prototyping process.


Collaboration is a key theme in the video, with the emphasis on how AI can enhance the collaborative process among designers. The script talks about the potential for AI to assist in understanding context, communicating across roles, and discovering important work within a team's design process.

💡Genius Chat

Genius Chat is a concept introduced in the video as a direct interactive experience with AI within the design tool. It allows designers to communicate their needs to AI through text, which then provides suggestions or makes changes to the design. This represents a more conversational approach to AI integration in design tools.


Jenny Wen, a design manager at Figma, opens the session by discussing the importance of AI in the design industry and Figma's focus on it.

Noah Levin, VP of Design at Figma, introduces the concept that design is about solving problems for people and how AI can assist in this process.

Levin emphasizes the historical evolution of design with technology, suggesting AI is the next step in this progression.

AI's potential to change the design process is discussed, with a focus on how it can handle complex interfaces and task-based computing.

The idea of shifting from an app-centric world to dynamic task-based computing is presented as a future possibility due to AI advancements.

Noah outlines the potential for AI to change what is designed, how it is designed, and who gets to design, focusing on product, process, and people.

The talk introduces the concept of AI simplifying design work, allowing designers to focus more on strategy and taste rather than minutiae.

The importance of design systems in preparing for the integration of AI into the design process is highlighted.

AI's role in democratizing design by lowering the barrier for non-designers to participate visually is discussed.

Figma's community-centered approach to developing AI tools that serve user needs is emphasized.

The acquisition of Diagram by Figma is announced, signaling an exciting development in AI integration for design tools.

Jordan, Vincent, and Andrew from the Diagram team share insights on the future of design with AI and their journey in creating a design assistant.

The evolution of design tools at Diagram, from automator to prototyper, is outlined, showcasing the progression towards AI integration.

The challenges of representing visual Figma canvases to a language model and the solution of converting designs to webcode are explained.

The concept of Genius Auto Suggest is introduced, where AI provides design suggestions in real-time within the Figma canvas.

The development of Genius Chat is revealed, allowing for a more direct conversation with AI for design assistance within Figma.

The Diagram team invites community feedback on their AI tools, emphasizing the importance of co-creating the future of design with AI.