On the Road to ITK Facilitation: Practice & Feedback

On the Road to ITK Facilitation: Practice & Feedback

Today’s post is co-authored by Bill Donaldson and Lauren Armbruster

ITK Trainee’s Perspective:
The road to becoming a certified Innovation Tool Kit (ITK) member is clear and full of community support-but it can also make the candidates a little nervous! For instance, one requirement is for all facilitators-in-training to run facilitation sessions while being observed by a veteran ITK member. Practice makes perfect…but I don’t think I’m alone in saying that we all wish we could be perfect on the first try! Thankfully, ITK members are trained to think positively and provide constructive feedback-which is exactly what I received from one of my observers, a certified ITK facilitator.

My experience with observation during an ITK practice was simple: I ran a Lotus Blossom brainstorming session as the sole moderator, Mural operator, and facilitator. I thought that the audience was engaged and able to meet the goals we had established in advance, plus have a little fun along the way-but I was looking forward to hearing what a more experienced facilitator might suggest as best practices for the future.

Team Toolkit Member Perspective:
I saw a post asking for someone to be an Observer during a Workshop through our internal channels. Knowing how hard it was for me to get those last items for ITK Certification, I was thrilled to help. “What an opportunity to help her get closer to her ITK certification”, I thought. Never giving it much though how I would actually do this, after all I’m a certificated ITK facilitator.

While in the session taking awesome notes, it came to me “how am I going to give her feedback?” So right after the end I did that ITK thing, improv.

I started with the COIN model, but wanted it to provide more of a critique than feedback. I used the Critque method (“I Like, I Wish”) from an ITK Lunch and Learn. This still didn’t convey what was accomplished by the ITK Trainee in her Workshop, so I then applied the ITK superpower of “AND” adding a User Journey on top like a ‘cherry’ on a sundae.

What is this sundae? At the base three columns: “I Like” and “I Wish” and “I Hope” Using “I” is a lighter way of providing feedback. Making the feedback specific things heard, seen, and felt. The User Journey is a great way to provide how things changed over time for the specific observations. But wait there’s more… all of this can be funner [yes this is a word]. Add icons, pictures, or even use a book or movie theme like to describe the Workshop. Humans are story sponges, our brains are wired to hear and remember stories. So make your observations reflect the story you saw and heard.

The recipe to recreate it…

1) Start with a quick meeting (15 mins) before the Workshop with the ITK Student for introductions and to get any specific areas for feedback. These areas can be used to plot on the User Journey. Make sure to ask how does the person like to get feedback. Even very well done feedback can be a surprise if not expected. Likewise, skipping this can make it hard on you to provide meaningful feedback when things appear to go awry.

2) During the Workshop record Observable events and the Impact on the Workshop (both the flow and the attendees) with the specific Context. These are from the COIN model

3) After do a scan for themes and group into three groups: “I Like” “I Wish” and “I Hope”.

  • The “I Like” are that things your liked or even better when someone said they liked it during the Workshop.
  • “I Wish” should be gentler then “areas of improvement”. When “you wish” something happened, it’s just that you wish. As the receiver I can grant your wish or not.
  • “I Hope” give you the chance to push for something more or bigger that you see the observed person may not. You can make the case “I Hope” is the same as “I Wish” you wouldn’t be wrong. For me it was about the visual weight and appearance of 3:1 positive to negative comments.

4) Let the Story or theme come to you. Then find icons or pictures for observations and decorate the timeline. The entire effort should take under 30 minutes.

5) Meet with the ITK Student to present the Mural and let the Trainee define their Next steps. Start with asking the ITK Student how they thought they did. Use the exact words they gave you in step 1. Then go over the areas that aligned with their self-feedback and where you saw something different.

ITK Trainee Perspective:
I found this to be the easiest feedback session I have ever experienced at MITRE! I knew (somewhat) what to expect from it thanks to our expectations-setting conversation where I could identify specific areas that I was worried about or interested in them paying attention to. The timeline is a visual and intuitive way to track the progress of a full session. For me, this was most helpful to review very soon after I facilitated so I could remember my experience compared to what was observed and see if there were differences-and ask myself why that might be. Beginning with positive reinforcement through the “I Like” was a comfort and built me up before the more constructive “I Wish” and “You Might” feedback-but all of it was ‘light’ and easy to take and talk about thanks to the fun icons, memes, and imagery that were used throughout.

Overall, this form of feedback was easy to understand and a great mix of positive feedback and constructive criticism. It is highly recommended as a model to provide feedback to others within ITK and across MITRE after sponsor briefs, during annual feedback sessions, and beyond.

Getting The Most Out Of Hybrid Workshops

This week’s post is by Josh LeFevre, Julie Williams Stogner

COVID-19 changed the way we all collaborate. As individuals and organizations make decisions about the future of their workplace, we are likely to see more collaboration sessions with a blend of in-person and remote participants. While Josh’s personal preference is to host workshops that are all in-person or all virtual, we’d like to share what we’ve learned from a recent hybrid workshop we led.

This should be the beginning of the conversation around how to make ITK, design or innovation style workshops most effective in the growing hybrid collaboration space.

In May, Julie asked Josh to develop and facilitate a workshop to help improve decision advantage for logistics teams that are stood up in times crisis . Throughout June and July, Josh and Julie worked closely with the sponsor to craft a half-day in-person workshop with a series of activities to achieve the desired outcomes.

The sponsor asked all the participants to attend in-person to elevate the level of collaboration. One week prior to the planned workshop, some participants expressed they could no longer attend in person or that they only felt comfortable attending virtually, due to COVID-19. While this abrupt change was not ideal, we quickly pivoted to run a hybrid workshop with six in-person participants and four virtual participants.

As one participant observed, the workshop turned out great “due in large part to the upfront planning and exceptional team support.”

If you are going to lead a hybrid ITK session, consider using this brief checklist:


  • Test all connections, tools and tool access (like MURAL) for participants, video feeds, microphones, Teams access, projectors/monitors, etc., are working properly and are in sync to ensure virtual portion of the meeting will not have technical difficulties
    • Leave enough time to troubleshoot (do not wait till the day of to test)
  • Ensure all materials are printed or available online ahead of time
  • Have one lead facilitator and at least two – four co-facilitators (3-5 facilitators total)
    • Lead facilitator should attend in person if possible
    • Have at least 1-2 extra co-facilitators in-person to collect notes and update lead facilitator of incoming virtual feedback. One of these facilitators should be responsible to monitoring the online chat and video feeds in the room.
      • The secondary facilitator may be needed as escort for participants depending on site guidelines
    • Have at least 1-2 facilitators online to capture notes, transcribe in-person activities and manage online conversations. One of these facilitators should be confident running a virtual workshop on their own in case there are technical issues.
  • Make “video on” a requirement for virtual participants (or at least when they speak and participate). Video on encourages engagement and less distraction of virtual attendees. This also helps in-person participants feel as if they are being eased dropped on.
  • If using MURAL (digital whiteboard) and in-person whiteboards, make sure both are updated and match synchronously
    • This can be done by either an in person or online facilitator
    • This task is incredibly challenging as those attending virtually cannot always see the activity outputs until they are completed
      • Encourage participants to read their contributions as they are added
      • Facilitator (Lead) can re-read the up-voted contributions
      • Secondary facilitator can take pictures after each activity to ensure the online participants and facilitator(s) are able to follow
    • Take time to pause and invite virtual participants to contribute to discussions
    • Take breaks after dot voting to tabulate online and in-person votes
    • Consider utilizing an MS Form version of Rose, Bud, Thorn
    • Use printed handouts and virtual handouts for attendee bios, instructions for site, instructions for activities, contact information for follow on discussions (create these ahead of time)


  • Have a smart projector or large touchscreen up in the room for in-person and remote persons to collaborate seamlessly via a digital interface
  • Provide video feeds via tablet or mobile device at every in-person working group to facilitate a stronger in-person/remote collaboration during discussions
  • Pair up one in-person participant with a remote participant, throughout the workshop, to ensure the remote participant’s voice is heard and has a clear access to provide contributions to mixed (in-person and virtual) breakout groups
  • Take more frequent but shorter breaks to ensure remote attendees stay connected and to sync in-person and virtual contributions, as needed
  • Conduct a “how to” MURAL or platform training ahead of the hybrid workshop


Introducing The Time Blossom

Introducing The Time Blossom

Today’s blog post is by Allison Khaw

Let me tell you about the Time Blossom!

This innovation tool can be used to help you better manage your time on a project. On a larger scale, the Time Blossom is one of several existing “X-Blossoms,” which are specially-themed variations of the Lotus Blossom. Inspired by the diversity of X-Blossoms created before me, I chose time management as the underlying theme for the Time Blossom because of how pivotal a role it plays in our busy world today.

First, a refresher—to use the Lotus Blossom, start by labeling its center box with a problem or topic of your choice. From there, leverage the tool’s grid structure to brainstorm ideas and propagate them outward to the surrounding blossoms. These ideas may be characteristics, categories, or solutions related to your central topic, but the key is that you’ll generate many of them in a rapid manner.

The Time Blossom has the same structure as its parent tool, but it is pre-populated with eight open-ended questions that revolve around time management and prioritization. Thus, to use the tool, select a project or subject area to focus on, and then fill in the outer blossoms with your answers in the context of your project.

It’s as simple as that.

In particular, consider the top row of questions in the Time Blossom. Think of these as the “Goldilocks” questions, but in units of time: they ask you to identify tasks that are running “too hot,” “too cold,” or “just right.” In fact, the question, “What are we spending the right amount of time on?” was not included in an older version of the Time Blossom; only after early testing did I discover its importance. Recognizing what we’re doing right is just as valuable as recognizing what we can do better, after all!

A 9x9 grid of colored squares with words in most squares

Let’s take a look at a Time Blossom example. Imagine a fictional scenario where you and your team—Alisha, Gabe, Kai, Liz, Pat, Rob, and Stacy—are holding a brainstorming session on the development of your product. (This realistic premise stems from our very own choose-your-own-adventure-style book called The Toolbox of Innovation!) If you populated the Time Blossom as shown, what might you conclude about how effectively you are managing your time?

Perhaps your team would conclude that your product development is going well but could still be improved by the reprioritization of certain tasks. For instance, you might plan to spend more time with users or set up a meeting to uncover any hidden assumptions about your product. The boxes with thumbs-up symbols indicate the ideas that especially resonated with the team. Down the line, you might even hold Lotus Blossom sessions for some of the more complex ideas that should be further explored.

To me, improving your time management skills is as much about increasing your self-awareness as it is about working efficiently. Knowing which tasks you’re prioritizing, and why, can give you a sense of clarity that is hard to find otherwise. Tools are just one way to help with that. The Time Blossom is by no means the first—nor will it be the last—time management tool, but if it prompts you to reflect on these topics, it is time well spent.

Download an editable Time Blossom template

The D4 Approach To Innovation

This week’s post is from Conor Mahoney, one of our new ITK Facilitator candidates

Every morning when I fire up my laptop, I often take a few minutes to peruse the latest postings on Arxiv, SSRN, and ArsTechnica, to ensure that I stay up with new technological developments and research findings. As a data scientist, a field that is both rather nebulously defined and constantly evolving, this is a task that often leads me chasing new “white rabbits” down their intellectual warrens like Alice from Alice in Wonderland.  As I fixate on a new shiny object my thoughts immediately turn to my Sponsor’s programs, like someone working on a jigsaw puzzle with missing pieces, I see this new widget or tool filling in some gap in capability or function and greatly enhancing the end result.

However, successful innovation efforts rarely begin that way, with a technology in search of a use case. After 22 years inside the federal government and military ecosystem, I have yet to see this approach work. Instead, the majority of the successful innovation efforts I have participated in started by deconstructing an existing problem and identifying areas where technological injects or process refinements could alleviate user pain or deliver enhanced results. It can be difficult to resist the siren call of the shiny and new and focus instead on what really delivers value: enhanced capabilities tied to well documented user needs.

To that end I developed a phased approach that focuses teams on realizing and ameliorating problems, using several of the ITK tools. I call it the D4 approach to innovation, and it involves Discovery (identifying mission or capability problems) Deliberation (Identifying what pain points or capability gaps to address based on the perceived value gain), Deconstruction (unwrapping the problem to identify its threads, touchpoints, and relationships) and Development (ideating on solutions). I believe that one of its best value propositions for this approach is its ability to familiarize “outsiders” to a project and the problems looking to be addressed.  Bringing in people from outside fields, mission areas, and organizations to look at problems with new perspectives and fresh approaches can often lead to excellent, outside of the box, ideas and it increases the diversity of thought that can be used to attack a problem set.

In terms of ITK tools this process can be thought of roughly as: Define, Understand, Generate, and Evaluate.  In the Defining phase I might use Problem Framing to develop and document the problem statement at hand. With a group consensus on a problem statement, I can then Journey Map out the problem against a user or system perspective. As I move to Understand, I might break out the Triz Prism; which flows in sequence because it focuses on a well-defined problem (which I developed over the previous phases) and ends on a specific solution. Finally, during the Evaluate phase I like Rose, Bud, Thorn to identify potential issues with our specified solution or enhancement and to consider potential negative impacts to other processes or related capabilities.

There is, of course, no wrong answer when it comes to tool-chaining across the ITK tool palette. But putting tools end to end that build on one another is a useful approach that provides for continuity of thought and focus and it can help ensure that the innovation session never gets too far off the rails. Thinking about the tools as part of a larger, integrated, process and planning their use beforehand will often also provide “parking lots” for when team members rush towards a solution without defining a problem. I hope you enjoyed reading about my approach, and I hope you have fun innovating out your own.

ITK & Health Leadership Development

This week’s post is by Tracey Amos

In early May, MITRE’s Health Leadership Development team partnered with the Innovation Toolkit (ITK) team to deliver a weeklong Innovation Toolkit training series focused on hands-on use of several of the most popular Innovation Toolkit tools – Lotus Blossom, Mission and Vision Canvas, Problem Framing and Premortem.[1]

The series kicked-off with two 1-hour ITK overview sessions. The overview sessions drew over 200 participants and based on feedback, provided a valuable introduction and insights into how to use the Innovation Toolkit tools. In the words of one participant: “I had only heard about the ITK but didn’t really understand what it was. The session was a great introduction to the set of tools.”

The kick-off event was followed by five days of tool specific training. Two of the sessions were structured around use cases for current MITRE projects.

One session focused on the Premortem tool training: Project Leader Iris Sherman partnered with ITK facilitators Gabby Raymond and Jonathan Rotner to build a use case exploring possible impacts of not implementing a MITRE program recommendation in its final report. Using the Premortem tool, the training session framed and explored a worst-case scenario and used the input to define an updated goal and risk mitigation strategy. In the words of one participant: Instead of waiting until the end of a project to find out what went wrong, and learn for the future, we can use this technique to go on an “imaginary time travel” to avert real failures.”

The other session focused on  the Problem Framing Tool: Project Leader Audrey Winston partnered with ITK facilitators Tracey Amos and Tammy Freeman to build a use case to examine how Robotic Process Automation (RPA) and Artificial Intelligence (AI) might be applied to increase the ability of Medicare website visitors to complete routine tasks or search information without costly human intervention and to understand how other organizations serving similar populations (e.g., senior citizens, disabled, caregivers) use RPA and AI. The training session provided an opportunity for the MITRE project team to engage in a collaborative conversation with others, broadening their perspective around areas where automating processes using RPA and AI could potentially be a “game changer.” The team is confident that a compelling case can be made for further exploration in this opportunity space.

The Innovation Toolkit Training week ended with a “Meet the Authors” Roundtable event where members of Team Toolkit engaged attendees in discussion about the inspirational journey to curate the ITK suite of tools and to discuss their recently published book, The Toolbox of Innovation, a lively, playful gamebook written in a’ choose your own adventure’ style.

The Innovation Toolkit Training week was part of a suite of FY21 leadership development and training opportunities sponsored by Health FFRDC leadership as part of their commitment to “building trusted and innovative leaders.”

Throughout the course of the week, participants expressed interest in learning more about the ITK tools and how to apply them to engage their teams and their sponsors. In response, several Health FFRDC “pop-up” ITK events are being planned for later this summer! Stay tuned.

Has your project also used the ITK? Please comment and share your story!