The Examine team just got back from our annual retreat. The retreat was in Mexico because it can be quite difficult for some of our team members to get a visa to the U.S. or Canada.
In line with our dedication to transparency, I figured I might as well tell you exactly what we talked about and how the retreat went so you can know where Examine is headed.
Day 1: Flying in and reacquainting
Here’s most of the Examine team, in a “silly pose” photo. (We took a regular one, too, but I doubt you’re interested in that.)
We see each other only once a year, so we have to get used to getting to know each other in person, and some of us can be different from what we’re like online. For example, see the guy in the back row? His name is Peter. He’s a quarter-inch square avatar online, but 6’9” in real life.
When meeting up, we found that everyone’s twice as silly in real life, and that I’m at least five times sillier. Most importantly, the Examine team is weirdly kind. I’ve never met a kinder group of people.
About 80% of the team are researchers or clinicians from a wide variety of backgrounds: pharmacy, medicine, lab research, dietetics, etc. The rest of the team does literally everything else — tech, customer support, marketing — that Examine needs to keep the lights on.
Not everybody had met each other before. Luckily, I love thinking of contrived situations and telling people about them, and I was able to use this skill (or some would say weakness) to make a weird icebreaker for the Examine team. They split into groups and competed on a fake show called “Lemur’s Lair” — similar to Shark Tank. Except this show was weird — it had business ideas, but also plank contests, typing contests, and stare-at-each-without-smiling contests.
Anyway, I digress.
The place where we stayed had howler monkeys living in trees outside some of the rooms. Monkeys are super cute, but they also like to communicate murderous intent and wake you up in the middle of the night.
By the way, did you know that peacocks make loud noises that sound like a colicky baby wailing? I’m a light sleeper due to having bad joints, so I switched rooms twice in the first day and a half. Sol, my co-founder, was bitten so often by mosquitos in his room that his pillow had multiple blood stains on it.
After we settled in and got better sleep, we dove head-first into some complex discussions …
Day 2: Examine weaknesses and artificial intelligence
During the first day of meetings, we collected a mountain of feedback from the team about Examine’s biggest weaknesses. We settled on 11 main weaknesses (that sounds like a lot when I type it out), and spent much of the first day talking about one of them: artificial intelligence (AI).
In a nutshell, AI is potentially both an existential threat to Examine and an awesome way to improve Examine.
AI tools are pretty okay at giving answers to well-understood and settled-upon health topics. But in our internal testing of health-related AI tools, they too often make stuff up (or “hallucinate”, as it’s called) and convey certainty in areas where there is none. Also, they can’t keep up with the latest research or tie disparate research together.
But many people are just looking for quick answers. If everyone gravitates toward quick answers that may or may not be correct, Examine is in trouble. We don’t think that will happen though, because verified health information is really, truly important. Bots are great until they give you dangerous or outdated health advice, and bots can’t yet tie together advanced study methodology assessment with expertise in clinical and research fields.
AI’s potential usefulness to Examine is two-fold. First, if an Examine bot is trained on Examine data, it can give you quick answers without incorporating questionable information from elsewhere on the web. That means it won’t be reading health guru viewpoints, inapplicable studies, and marketing claims.
Second, if an AI tool could help us extract data from studies even 10% faster, that would really add up over the thousands of studies we analyze. So far, the tools we’ve tested haven’t been reliable enough to achieve these two goals, but with the technology rapidly improving, we’re optimistic.
Day 3: Wonky search bar, reader feedback
Our third day was much simpler. We ranked the top 11 Examine weaknesses and found that two stuck out.
First, it’s not that easy to find information on the Examine site, and the information is often in more than one place, which is confusing.
There’s some low-hanging fruit that we can fix. For example, just yesterday we added labels to the search results so you know what kind of page (condition, intervention, outcome, etc.) you’re clicking through to.
In the next few days, we’ll add filtering to the Examine Database to make it even easier for you to find exactly what you’re looking for.
Over the next months, we’ll pick higher-hanging fruit, but we’ll need your help with that. Examine needs to do a better job finding out what our readers ideally want, and what they’re having a tough time with — both on the website and with their health conditions and goals — so we can match it up with what we can realistically do. We send surveys on occasion, and do phone interviews as well, but we don’t have a systematic and regularly-scheduled process for feedback. Time to get moving on that!
So if you’d like to volunteer to answer some questions and give Examine feedback on occasion, just contact us here. You won’t be badgered with a bunch of emails, and you can stop whenever you like. These won’t be generic corporate questions — they’re specific questions we came up with that will directly affect the future of Examine. For example, “Do you like this previous version of the Examine Database better, or the second version?”
Day 4: Free day
Our meeting days involve hours upon hours of intense discussion, so we scheduled one free day for the team to relax and not have to discuss and debate all day.
Some of us went into town to find delicious birthday cakes. We celebrated a couple team birthdays with tres leches and caramel.
When you just talk about work all the time, it gets a bit stale. Taking a break can help you think better, especially if the break doesn’t involve mindless web or social media scrolling!
Day 5: Data management
Long story short, we have a ton of data on the site and we needed to review our data management options.
If we’re able to accurately prognosticate and choose the best option, we would be able to significantly cut down on the amount of time it takes us to update pages, create new ones, and fix mistakes. If you don’t care to read a recap of this discussion, skip forward to the next section of this email!
Imagine you have a database of thousands of studies and variables extracted from those studies, as well as over a thousand web pages containing select data from the studies.
What’s the best way to manage this kind of data while making it easy for researchers to update material, get assigned tasks on the fly, and communicate with reviewers and copyeditors?
For the past 12 years, we used our own in-house software for much of that. But Examine is a small company, and we don’t have nearly enough money to build a sophisticated system that perfectly fits the above scenario. We have a massive amount of data to manage, but we need to spend most of our budget on researchers rather than tech, because our readers depend on our research.
There are a bunch of third party tools that are half spreadsheet and half database, along with other tools that are great at project management. No single tool does everything we need, but some of them are much better at an aspect of data management than the system we built.
We did settle on a few improvements going forward, with more discussions scheduled. Basically, we’re optimistic we can improve. We have not been updating our pages as fast as we want to, and getting our pages updated is our #1 priority.
Day 6: What’s next?
There are a bunch of cool features that we could add to the site. Every cool feature takes time and money though, so we have to balance them against alternate uses of that time and money.
On day 6, we ranked some of these features, so that we can start figuring out exactly how hard they would be to implement. The end result should be a site that you feel like visiting more often, that has information more specific to your situation, and that’s different than literally every other health information site out there.
So while Examine is already a different type of site — tinier than the big dog websites, but way more nerdy and objective — we’re going to innovate faster than ever before.
If you read this far down: thank you! I’m extra glad you’re on this ride with us, and I specifically want to hear from you. If you’re okay receiving some occasional questions from me about Examine features, just contact us and let us know!
Sincerely,
Kamal Patel
Co-founder, Examine