Background
Jason Charnes is Staff Product Developer at Podia. We talked to him about Podia's internal workflow for building their email editor product, the benefits and trade-offs of testing in Litmus, and navigating the complexities of email rendering as a developer.
Questions
- Tell us about your role and what you work on. What led you to need a tool like Litmus? What else did you evaluate?
- Can you talk about the workflow of using Litmus alone and with your team? What features do you make use of?
- Sounds like the copy and paste workflow could be potentially really annoying. Don’t they have a code editor and different plugins where you can just write code directly in the browser and have it render? Did you consider using that? If not, why not?
- In this compile step, let's say you wrote some high level HTML CSS, and then you have this tool that compiles it down, so it puts all the styles inline and that kind of thing. Then you copy and paste that over to Litmus, and see what it looks like. Would that affect how you write HTML CSS?
- Can you talk about the pricing plan you were on and and then, your thought process around it?
- Was it very annoying or just slightly annoying to be like, "Hey! Let's take fewer screenshots because we’re coming up on our limit"?
- Now that you're done with the project, have you canceled Litmus? Or are there hooks for ongoing monitoring, testing that you think are compelling or interesting?
- What about some of these features—ongoing QA kind of things—where maybe you can automatically forward one out of every 100 emails that gets sent?
- There's this wrinkle that we talked about earlier, which is that you're not just an email developer or designer where you've coded a single template, you have built a template builder and there can be a lot of different variations of different emails. Do you think that that resulted in higher Litmus usage that you had to test more variations? Or, on the flip, was it more integrated into the actual product and platform that maybe your Litmus usage was actually less?
- As an email developer, you're making it pixel perfect and trying to optimize across as many browsers as possible. But when you're building this platform, you still want it to make sense. What’s your take on hyper optimization vs. getting it generally correct and having the right frameworks?
- This reminds me of SEO in that way. It's like, "Hey! There are SEO experts and all they do is dig into the internals, but it's nice to have a score and to just hit the big red flags.”
- You mentioned that maybe you're out of the target audience. Can you talk about, based on your perception of using Litmus, who you think falls squarely within their target ICP?
Interview
Tell us about your role and what you work on. What led you to need a tool like Litmus? What else did you evaluate?
I’m a Staff Product Developer at Podia. I work on a lot of early prototype stuff. For the email project, I was responsible for the actual email editors, so I worked very closely with another developer to take what was very basic editing before and build an actual almost WYSIWYG experience—an email building experience with styling, stuff like that. I oversaw the whole email project towards the end, but that was my main technical contribution.
Email is notoriously terrible at looking good, well, looking consistent. In my own side projects, I'll avoid any type of email design if I have to. Because we were going all in on an email builder, it was important that it work and look as consistent as it could in as many clients as possible. It's notoriously inconsistent between—even within the Microsoft Office Suites itself—in fact, some of the earlier versions of MS Office don't support certain styles or HTML features, but even Office 365 and the desktop, like Clash and what they support and Apple supports everything. Then you have Google, which doesn't support SUGs. It's very tough.
So while we can read about it, we're never going to catch it all because most of us work on Mac. Even Gmail Android, Gmail iOS apps render differently. For us, in order to feel confident in what we were shipping, we needed to see it in old versions of Outlook and IBM's Lotus Notes, and things like that. We weren't trying to go for 100% support across the board, but we wanted good enough. We needed a tool to do that.
In terms of choosing Litmus, it has the market share. That's just the tool I know to use. I will say, with Litmus, it has a ton of features, and we used just a very small subset. We may have Googled it, but if we did, it was 20 seconds. It's like, "All right, well, Litmus is still the winner here." It's similar to a tool we use called Browser Stack, which lets us see how our sites look in different browsers, and so, it was similar content for me that I could paste some HTML and hit a button and it would generate screenshots for all of it. Plus being that this would save us a lot of time and a lot of resources.
Podia is like, "Yeah, we'll pay for it," which made it easier for us to adopt too, because they're willing to use it.
Can you talk about the workflow of using Litmus alone and with your team? What features do you make use of?
Litmus has, as I mentioned, a lot of features and to me, it seems like they cater to ongoing email testing and that's not necessarily what we needed it for.
The workflow was basically what we would create—I don't know what they're called— maybe a “project” or “create an email,” just a single one. Then, we would actually skip the first two steps. We'd go to the design step and hit Open Editor and just take the raw HTML that we got after compiling the email, putting it into Litmus, then clicking Save and Preview, which would then generate all the screenshots for us.
After a while we could tell, it helped us solve some pretty big bugs. It also helped us decide which ones we weren't going to support, because no matter what we did, it was too antiquated, but it was just a lot of back and forth between those two steps—make a change, preview it.
Sometimes it would be selecting less screenshots, less different environments, because they charge you per screenshot. So one preview email would use 100 credits and by 10 emails you've used 1,000 credits. That was like, "Oh! We just hit our first month in a few minutes." So, over time, we generated a list of screenshots, only trying to pick out what we knew we needed.
They had a feature, I think we may have used a couple of times where you can forward an email in or send an email to them and they'll generate the screenshot. But that wasn't something we did very often because a lot of times, our actual workflow is to try it in development—just take it out of development and put it in there. It was a very tedious process. It made it easy to not Litmus check everything because it was so involved.
Sounds like the copy and paste workflow could be potentially really annoying. Don’t they have a code editor and different plugins where you can just write code directly in the browser and have it render? Did you consider using that? If not, why not?
We did not, because for our emails, we designed them in the most basic HTML and CSS we could, and then we ran it through a processor. We used Roadie. There's this tool that's pretty popular. It'll run your emails through this processor and then, this tool will inline the styles and take you to the CSS you've written. If it can convert it into inline styles, which has the most wide support for emails, it will do that.
So a lot of the time, the markup we were working with wasn't even necessarily the markup we generated. It was markup that a machine took our stuff and regenerated it, so we could test different ideas and theories in the code editor. But anytime we're in a process like that, the more changes we make, the less we remember what we've tried, what we haven't, the more we deviate from what we actually started with. So, it's good for prototyping, but for us to really know that we have something fixed, we have to go back to the process, go fix it, compile it, and paste it.
In this compile step, let's say you wrote some high level HTML CSS, and then you have this tool that compiles it down, so it puts all the styles inline and that kind of thing. Then you copy and paste that over to Litmus, and see what it looks like. Would that affect how you write HTML CSS?
Really, what the tool is doing is just if you have a style tag in the head of the email or anywhere in the email, or if you have a style that has a class name, what it's going to do is then look for all the elements that have that class and try and take as many styles as it can and inline them on each one and then, delete that class.
We found that that approach works for us because we're trying to support these older clients. We were a little forward thinking and making sure there's a tool called, Can I Email, which is like Can I Use for the web? But this is just for email.
We referenced that like the Bible almost. For example, we have social icons and well, usually on the web, we use SVGs for those, but we can't use SVGs because these four clients don't support it. So that guided our decision to use PNGs. Since we were making decisions like that, we found that most of the styles we wrote could be inlined. So, if there was a display issue, it was typically a matter of changing the CSS more than it usually was the HTML. We also went really table heavy in certain places because that's what you have to do with emails, unfortunately.
Can you talk about the pricing plan you were on and and then, your thought process around it?
I'm logging in right now so I can tell you. It looks like right now we're on the basic plan, which is $99 a month. It was two of us doing the testing, so we didn't need five pool users or anything like that. Since it limits you to one user, we just shared it.
We didn't use the design library or any integrations. We did use the code and visual editor, but then you add 1,000 email previews a month, and that was where we were about to have to bump up. But then we realized that the less we take, the less credits it used. However, no one within the company really batted an eye at $99 for that tool because we wanted emails to look good and we wanted them to be right. If that worked, then let's do it. It wasn't even really a question.
If we needed it, we probably could have gone higher, but we found ways to work within the constraints and they were fine.
Was it very annoying or just slightly annoying to be like, "Hey! Let's take fewer screenshots because we’re coming up on our limit"?
I was frustrated when I realized that every time I ran an email, I was draining our credit. That was frustrating because that wasn't clear. I was under the impression, "Oh! I'm generating a screenshot of this email and a bunch of clients." I was like, "Got plenty of room. We can do this 1,000 times."
It was mildly annoying that we had to generate less because Litmus almost has too many email clients for you to choose from. Maybe for some people that's what they need. For us, we only cared about a subset, really. Obviously we wanted to look good in all modern clients, and then I recognized that Outlook was something a lot of people still used.
I referenced it earlier, but we didn't make Lotus Notes look good. That wasn't our priority, so it was almost more of a pain to have to go back and uncheck all the ones.
In order to save money, I had to make a decision right then, “Which ones don't I care about?” And even then it felt like a guessing game because there's 40 or 50 options that may even be before the mobile options. That may just be desktop options. It's a lot, so yes, it was annoying. I feel like I had to do more work, but that's what I had to do.
Now that you're done with the project, have you canceled Litmus? Or are there hooks for ongoing monitoring, testing that you think are compelling or interesting?
We haven't canceled it yet. We've used zero out of 1,000 previews, and of course, I guess that reset yesterday though we have some ongoing projects within email that it might be nice to have. But at this point, I don't know that I can... Even though they were quick to be like, "Yeah, let's get it," I don't know that I can justify being like, "Yeah, let's keep paying for it," because I don't know that we'll use it in earnest as much as we did when we were getting started.
What about some of these features—ongoing QA kind of things—where maybe you can automatically forward one out of every 100 emails that gets sent?
We did that in our beta. We had somebody take a picture of their phone and it was like, "Can you just forward us that email?"
We took that through Litmus. So it definitely would be useful in those cases, but we're probably looking. If I had to guess, even with a lot of the email features we're working on, they also aren't necessarily about the email design themselves now. That prioritizes it even lower. So it would be useful to have—maybe not $100 a month useful—but it was useful for the last six or seven months.
There's this wrinkle that we talked about earlier, which is that you're not just an email developer or designer where you've coded a single template, you have built a template builder and there can be a lot of different variations of different emails. Do you think that that resulted in higher Litmus usage that you had to test more variations? Or, on the flip, was it more integrated into the actual product and platform that maybe your Litmus usage was actually less?
We're probably, to be honest, not Litmus' target audience. So, I think building an email builder, we were having to test a lot of things, which was actually helpful because even though we were testing different templates, under the hood, they were all the same mechanism for generating solves and stuff.
So typically, if there was a bug with one, it's possible that bug could be created in another template. At first we were doing it a lot, but then, it felt like it was just getting smaller and smaller at the number of times we were having to do it.
As an email developer, you're making it pixel perfect and trying to optimize across as many browsers as possible. But when you're building this platform, you still want it to make sense. What’s your take on hyper optimization vs. getting it generally correct and having the right frameworks?
That's a good way to frame it. In a perfect world, we would hyper optimize, but it seems like that could be a full-time job of several people trying to get it to work. That universally and for us, entailed some friction though not necessarily in the product but just that we're web developers and we work with modern web technologies.
So there's a lot of things that have just become second nature to us that we could not use in email. That's where Litmus was helpful—Using the Can I Email and then putting in Litmus was very helpful.
There was something else that I think is worth mentioning and I forgot about. So you mentioned Semantics. It gives you a score of how good the market is. There's QA checks and it gives you check marks, things like that or what needs to be worked on. That was really helpful because, as someone whose job is email developer, I had a lot more confidence. It's not that we made it look good and it's on stilts behind the scenes, it's that we made it look good and according to this tool, it meets best practices.
I use another tool on my side projects called MailTrack, so I can send emails in development and it does something similar. It gives you a health rating. While it's not the selling point for me, it is useful. It's a confidence booster and a checklist for someone to know what's wrong and how to fix it. That's valuable.
This reminds me of SEO in that way. It's like, "Hey! There are SEO experts and all they do is dig into the internals, but it's nice to have a score and to just hit the big red flags.”
Exactly. I guess there’s like Ahrefs, I don't know how you pronounce it, but exactly. Anytime I put a site in there, it's like, these are all things you need to fix. I'll spend money on a tool that tells me how to make things right. That is valuable to me.
You mentioned that maybe you're out of the target audience. Can you talk about, based on your perception of using Litmus, who you think falls squarely within their target ICP?
I wish I knew. So I used Litmus in 2014 when I worked for an ad agency and they wanted to do a HubSpot campaign. We designed the emails and coded them ourselves from scratch. I felt like I was a better audience for that then, a better customer because I had, in the number of emails that we wrote ourselves, I needed to see how they fared across all those, almost like you said earlier, an email developer. That felt like a good fit. This did not feel like a good fit for what we were doing, but it still got the job done.
I feel like people maybe working in marketing, maybe even some non-technical people with the ability to forward an email can see how it looks because in theory, you could use this outside of writing your own emails if you built an emailing tool.
Disclaimers
This transcript is for information purposes only and does not constitute advice of any type or trade recommendation and should not form the basis of any investment decision. Sacra accepts no liability for the transcript or for any errors, omissions or inaccuracies in respect of it. The views of the experts expressed in the transcript are those of the experts and they are not endorsed by, nor do they represent the opinion of Sacra. Sacra reserves all copyright, intellectual property rights in the transcript. Any modification, copying, displaying, distributing, transmitting, publishing, licensing, creating derivative works from, or selling any transcript is strictly prohibited.