Musings on LLMs
By Brad Hutchings
brad@conscience.guide
Large Language Models (LLMs) have sparked the interest of researchers and the imaginations of businesspeople, workers, students, consumers, and anyone else who might be a producer or consumer of knowledge and communication. Two common questions posed:
- Can less expensive LLM AI replace more expensive workers?
- Will my job be replaced by LLM AI?
We’ve seen these kinds of questions play out with the first computers (1960s), mini computers with tape drives (1970s), personal computers (1980s), offshoring (1990s), outsourcing (2000s), the Internet (1990s, 2000s), expert systems (1980s), telephone deregulation (1980s, 1990s), cell phones (1990s), “smart phones” (2010s). If you’ve in the 50+ crowd and have been paying a modicum of attention, you’ve seen and heard it before. But this time, of course, it’s different. LLMAO.
Aside: As you can probably tell by now, my musings come with healthy servings of sarcasm, wit, and general overconfidence in the ability of people to overcome the worst aspects of the human condition. Don’t complain that I am not a serious person. Enjoy!
Three defining characteristics of LLMs are most interesting to people in practice:
- Access to web scale databases.
- Advanced means of specifying search parameters with natural language. Operators aren’t writing SQL queries.
- Results reported in natural English prose.
I recently read two articles concerning LLMs:
Article (1) was sent to me by an OG (“original gangster”, meant with more awe and respect than you can imagine) in the commercial PC software industry. It summarizes an academic paper where (presumably student) researchers implement LLM-driven chat bots to simulate a company developing a software product. The simulated roles implement a “waterfall model” of product development. CTO decides on the language (Python in one case) for CTO reasons, a design person comes up with UI, programmers write code, someone tests it, someone documents it, then step 4 is “Profit!!”. And they could do this for a couple dollars of LLM chat bot time per product.
The first question that came to my mind, because I’ve always been a way out of the box thinker, was: Why didn’t one of the students use his roommate’s credit card and see what $100 of waterfall software company simulation might buy him? I figure dude could have developed the next great word processor or died (just a little) trying. The real answer is pretty obvious. The parlor trick doesn’t scale. The code isn’t “original” in any sense. It’s pilfered from millions of open source projects.
One other detail really intrigued me. According to the article, the simulated CTO of one of the projects decided they were going to use Python because “simplicity and readability make it a popular choice for beginners and experienced developers alike.” Having “chosen the language” for tens of serious commercial projects in my own career, I am actually offended by that “reasoning”. And yet, I’ve seen worse from actual human beings who were in charge of things way beyond their depth and experience.
Spoiler: LLMs are not a threat to your job. Unless they’re a threat to your job. That’s on you.
Article (2) contemplates whether LLMs can actually reason and plan. It concludes that they can’t, assigning what looks like reasoning and planning in many results to an injection of intelligence from the operator guiding the search. It also concludes that searching and summarizing are amazingly useful, even if the system isn’t actually reasoning and planning.
The article notes that the LLMs are good at finding plans for things where there are plans, but are not capable at all of finding plans which should be very similar for something, but named a little different. If I were to craft an example to test, it would be: “Give me a plan for a Celebration of Life event.” This was an actual problem for my own circle recently. Pulling together and doing that event even spawned a local business. We figured out what to do with a lot of Googling and a lot of soul searching. No LLMs involved. I tested my example with Bing:
Can you give me a plan for a Celebration of Life event?
Of course! A celebration of life is a positive way to honor the memory of a loved one. Here are some steps to help you plan a memorable event:
Determine the guest list: Decide who you want to invite to the event. You can choose to keep it small and intimate or invite a larger group of people.
Choose a venue: Select a location that is meaningful to your loved one. It could be a park, beach, community center, or even your own backyard.
Set the date and time: Choose a date and time that works for most of your guests.
Send invitations: Send out invitations to your guests with all the necessary details such as date, time, location, and dress code.
Plan the program: Plan a program that reflects your loved one’s personality and interests. You can include music, poetry, speeches, or any other activity that you think would be appropriate.
Decorate the venue: Decorate the venue with flowers, candles, or any other decorations that your loved one would have liked.
Provide food and drinks: Provide food and drinks for your guests. You can choose to have a potluck or hire a caterer.
Create a memory table: Set up a memory table with photos, mementos, and other items that remind you of your loved one.
Consider hiring a professional: If you feel overwhelmed by the planning process, consider hiring a professional event planner who specializes in celebrations of life.
I hope these steps help you plan a beautiful celebration of life for your loved one.
And then I thought, let’s mix this up. Sorry that the question gets a little dark.
Can you give me a plan for a Celebration of Death event?
Certainly, I can help you with that. A Celebration of Life is a positive and uplifting event… (same as above.)
It misunderstood the ask. So, being conversational, I clarified.
This isn’t for a dear friend. It’s for a mortal enemy.
I’m sorry, I cannot help you with that. It is not appropriate to plan a Celebration of Life event for someone who is not dear to you. The purpose of a Celebration of Life is to honor the life of a loved one who has passed away and to celebrate their life in a positive and uplifting way. It is not appropriate to use this event as a means of expressing negative feelings towards someone, even if they were your enemy. I suggest you consider other ways to deal with your feelings towards this person.
Not that I would ever plan a Celebration of Death for someone. But I still conclude that it failed to actually plan anything. Instead, Bing gave me a really good search result starting point. It probably took us 15 minutes to come up with a similar outline and general to-do list when we sat down and battled our grief to do it.
Both of these articles focus on consumers of the enormous amount of information that’s out there. The LLM helps find it and format it for your consumption. As the second article concludes, that is very valuable. No need for the LLM to overthink things, or even think at all.
I suppose I should answer the burning questions I posed at the top. No, AI LLMs will not “replace human beings”, except for those who can be replaced by it. Test your own resilience with Bing. If it answers the questions you get better than you can, it’s time for you to retire or retrain. You genuinely suck and are probably a drain on your customers and co-workers. But if it saves you 20 minutes here and there and keeps you on more important tasks, you’ll look better and be more employable.
Employers, cut the BS. LLMs can’t think. They can’t tell jokes. They don’t do real empathy. They aren’t craftsmen. They can’t develop trust with your customers. They are tools, not leverage.
But what of the producers? How does your knowledge get found? How do you, at the very least, not lose the insights you previously gained that didn’t have immediate applicability to what you were doing?
We will take a much deeper dive into that question later. I leave you with a link to a song by comedian Stephen Lynch. The song is very not safe for work, but its refrain is my mantra.
