Working With AI: Two Paths to Prompting

Don't overcomplicate things.

Lots of folks have read our paper showing that using AI boosted the quality of the work done by consultants at the top-tier Boston Consulting Company by 40%, but there is a key factor in that paper that most people are missing. The consultants were not given some special version of AI, trained on proprietary data and with a customized interface. Nope, they were just given GPT-4 with minimal training and examples. The plain old GPT-4 from back in April, before all of its new capabilities were added. The same GPT-4 that everyone in 169 countries can access for free, via Microsoft Bing in creative mode. And while some of the consultants received a small amount of training (which didn’t help much), most of them just started using the AI without any instructions.

And they still saw massive performance increases.

The lesson is that just using AI will teach you how to use AI. You can become a world expert in the application of AI to your domain by just using AI a lot until you figure out what it is good and bad at. This is one of two reasons that I dislike the emphasis on prompting that pervades much of the discussions of AI: it makes using AI systems seem much harder and more mysterious than it is. Just use it and see where that takes you.

The second reason I don’t like the emphasis on prompting is that, for most people, having to worry about prompts at all is a very temporary state of affairs. As AI systems improve, the need for esoteric prompting decreases, because the AIs themselves become good at figuring out what you might want. My favorite illustration of this change is in AI image creators. If you didn’t already know, OpenAI released a new image creator called DALL-E3 (you can get it through paid ChatGPT Plus by selecting the DALL-E mode, or free through Bing). While it is not significantly better than other image makers (like Midjourney or Adobe Firefly), there is an important difference: rather than creating prompts for images, the DALL-E system lets you just talk to the AI about the art you want, and the AI creates the prompts for you. (...)

Conversational Prompting

For most people, you can just talk to the AI to ask for what you want. You can even talk to the AI if you don’t know what you want - just tell it what you might need and see what happens.

As a chatbot, the AI is really built for exactly this sort of use, where you speak with the AI as if it’s another person: the infinitely helpful graduate student who is a little naive and wants to make you so happy that they will make up facts rather than disappoint you. As you work with the AI using this approach, you’ll develop an intuition for what its limits and strengths are, where it is generally truthful, and when it is unreliable.

There is one major trick that will make your conversations work better: provide context. You can (inaccurately but usefully) imagine the AIs knowledge as huge cloud. In one corner of that cloud the AI answers only in Shakespearean sonnets, in another it answers as a mortgage broker, in a third it draws mostly on mathematical formulas from high school textbooks. By default, the AI gives you answers from the center of the cloud, the most likely answers to the question for the average person. You can, by providing context, push the AI to a more interesting corner of its knowledge, resulting in you getting more unique answers that might better fit your questions. Many of the more exciting uses of AI require this sort of specialization. 

The simplest way to do this is to start with giving the AI an identity (you are an expert, friendly teacher who helps students with complex topics). While that does not magically turn it into an accurate teacher, it helps give the AI the context of what types of answers you need, and what tone to use. You can also provide context in other ways, such as by pasting in the text you are working on, or a form you need to fill out, and seeing how it answers.

For most people, this is good enough to get started, and it is the technique I use most of the time when working with AI. Don’t overcomplicate things, just interact with the system and see what happens. After you have some experience, however, you may decide that you want to create prompts you can share with others, prompts that incorporate your expertise. We call this approach Structured Prompting, and, while improving AIs may make it irrelevant soon, it is currently a useful tool for helping others by encoding your knowledge into a prompt that anyone can use.

Structured Prompting

Structured Prompting is about turning the AI into a tool that does a single task well in a way that is repeatable and adapts to its user. Since the AI is not always built to do this, it will take experimentation and effort to make a prompt work somewhat consistently (it is very hard to reach 100% consistency with LLMs). To start, you need a clear goal. For example, in this prompt from our paper on AI and teaching, we want the AI to run a pre-mortem, an exercise where you image how a project might fail in order to decrease the risk of real failure. There are specific ways to do a pre-mortem, and, with the current generation of AI, there is value in ensuring that it is doing the process correctly, rather than just responding to chat requests. That is where the Structured Prompt comes in.

by Ethan Mollick, One Useful Thing |  Read more:

Image: Ethan Mollick

[ed. See also: What people ask me most. Also, some answers. (A FAQ of sorts).]