Project metadata | |
---|---|
Project title | User research for terrain.org: A Journal of the Built + Natural Environment. |
Project tagline | Improving engaged reading online through design |
Project summary | Usability, interviews, and a heuristic review |
Company/client name | Publisher, Terrain.org |
time frame | January-April 2016 |
My major responsibilities | User research plan, In -person user interviews in context, literature review on readability |
Platforms | Web |
Tools used | Pages, Keynote |
Key performance metrics | Increased readership, time on page, more donations |
Team | UX team of one |
Link to final project | Terrain.org |
How might we encourage readers to curl up with a screen the way they do with a printed book?
As a writer, you fight a war against indifference. You have to force people to care enough to click through to your story …they must answer a question… is this going to be worth my time?
–Quincy Larson, Free Code Camp
Project summary
Terrain.org is an online literary magazine that publishes high quality essays, poetry, and creative nonfiction about the natural and urban environments. This project aimed at increasing readership, encouraging more engaged reading of longer essays, and increasing donations.
I conducted usability interviews, and an expert review with other designers. My role was user researcher. I interviewed people in person and interpreted the results in collaboration with the publisher and site owner.
Executive Summary
At first glance, Terrain.org was attractive and looked like it was doing well. It had lots of visits (250,000 in 2015) and a large list of subscribers. But, when you went beneath the surface of visual design and images, there were some pain points. The number of subscribers had leveled off. Analytics showed the bounce rate was high (85%) and average time spent on site was low (less than a minute)–a major concern for a literary magazine featuring thoughtful essays, long articles, and poetry.
In addition, donations were low. The site does not show ads.
Pain Points
SESSION TIME: Average duration is less than a minute
RETURNS: Only 20% of the total viewers were returning visitors.
BOUNCE RATE: 85% of visitors exit from the home page.
PAGES VIEWED: The average number of pages viewed is 1.5.
In addition, the number of new subscribers has leveled off and few people use the “donations” link (less than 1% of site traffic). These numbers are worrisome for a literary magazine whose goal is to encourage long-engaged reading.
It appears that instead of browsing through the magazine, as people often do with print, visitors to this site generally read a single article, likely from a link. The challenge: how to keep people on the site reading horizontally once they click a link on one of the email newsletters.
It appears that instead of browsing through the magazine, as people often do with print, visitors to terrain.org generally read a single article, likely from a link, then leave.
Research overview
Since Terrain is an all-online journal and there are a lot of challenges to reading longer articles on screens, we started comparing how people read online as compared to print magazines. After a review of the literature on readability and surveying similar text-heavy sites, we conducted an expert review of the site’s usability. Next, we conducted live interviews and walk-throughs of the site with both current Terrain readers and potential readers.
Research questions
We wanted to compare the goals of new visitors browsing the site to newsletter subscribers who clicked through on an article.
- How does site usability and typography influence engagement with longer literary texts?
- How do Terrain readers arrive at the site?
- What are their motivations for coming to the site?
- What cues do readers use to search for content they want to read.
In addition, we asked people how they thought about supporting or subscribing to ad-free literary sites like this. Did they see the donate button? Were they aware that there were no ads? What would motivate them?
Data analysis
I created a matrix of results from the expert usability review (also called a heuristic evaluation) showing the scores, and comments for each check point, as well as an overall score for the site (54%).
In addition, the number of new subscribers has leveled off and few people use the “donations” link (less than 1% of site traffic). These numbers are worrisome for a literary magazine whose goal is to encourage long engaged reading.
Research overview
Since Terrain is an all online journal and there are a lot of challenges to reading longer articles on screens, we started comparing how people read online as compared to print magazines. After a review of the literature on readability and surveying similar text-heavy sites, we conducted an expert review of the site usability. Next we conducted live interviews and walk-throughs of the site with both current Terrain readers and potential readers.
Research questions
We wanted to understand the goals of new visitors and things like:
- How does site usability and typography influence engagement with longer literary texts?
- How do most Terrain readers arrive at the site?
- What are their motivations for coming there?
- How do they search for content they want to read.
In addition, we asked people how they thought about donations, to this and other sites. Is the Donate button too small to easily find?
Data analysis
I created a matrix of results from the expert usability review (also called a heuristic evaluation) showing the scores, and comments for each check point, as well as an overall score for the site (54%).
Expert review

User interviews
One of the main research questions was, “How do new visitors browse from the front page and find an article they want to read?” In other words, how do they stay on the site? To find out more, I combined the interview with a usability walk-through. I asked people to go to the front page, choose an article from the menus and read it to the end, speaking their thoughts aloud as they went.
I recorded the user interviews using audio and screen-capture videos. From our notes and transcripts, I organized the rich qualitative data obtained from the interviews by research question, using affinity mapping with sticky notes. I also noted new themes and ideas coming out of the research. I then organized this data into a Findings and Recommendations Matrix, featuring quotes, summaries and key insights from each participant.

Key findings
Most users commented positively on the site design, admired the images, and eventually read something interesting. Yet, the site’s overall usability score is below average. In some ways, the visual design created a barrier to browsing, finding, and choosing an article. The photographs were attractive and carefully chosen, yet generic in the sense that they usually did not come directly from the written content. They were distracting when used in the mega menus. Users lingered and engaged with the images rather than using them as indicators for choosing a written article.
Most people interviewed preferred print on paper for personal, literary, and in-depth reading over digital screens. Computers were seen as better for research, email, and finding facts. In keeping with this, participants all scanned the pages first, looking for cues to content before committing to an article. Everyone got stuck on the featured images, trying to use them to decide what to read, which was not reliable. The photos were either a beautiful distraction, or visual overload.
Everyone interviewed did finally enjoy some longer reading that engaged their interest. This supports our idea that providing the right visual cues, and clearing the critical path to interesting content would help people reach this desired state of engagement with text.
After watching people choose a story from the home page, I believe most of the page views come from links in the email newsletter to subscribers. People usually read only the page they clicked on, and don’t browse the rest of the site. That could be due to browsing usability. My theory is that a user is confronted with too many choices, however enticing, and has a hard time choosing just one to read. This is knowh as Hick’s Law: the more stimuli people have to choose from, the longer it takes the user to make a decision. In effect too much cognitive load and likely abandonment of the page.

Few people found the Donate and Subscribe links. People wanted to share articles, but there is no share link to make that easier.
Key recommendations. Usability improvements will help keep users on the site, but that alone will not increase readership. Since most people arrive at the site from external links to a particular article, Terrain should maximize its presence in the external places where likely readers readers would find the site, like links and banners on other relevant sites, content aggregators, social media, email newsletters, etc.
Given a longer-term trend away from home pages and towards apps and social sites, Terrain should look outward to publishing on other social sites like medium.com, Apple News app, and Facebook’s new Instant Articles feature which allows people to read without leaving that site. A longer-term goal might be to create a phone app, as more people are reading that way.
One recommendation that the publisher adopted from user interview data was the use of author photos instead of generic images. Here is an example of the new style from a newsletter.

A Readability Reading List
Balatbat, C. (2014, January 27). A Beginner’s Guide to Heuristic Evaluation (Part 2). Retrieved April 24, 2016, from http://blog.foolprooflabs.com/2014/01/beginners-guide-heuristic-evaluation-part-2/
Jabr, F. (2013, April 11). The Reading Brain in the Digital Age: The Science of Paper versus Screens. Retrieved April 25, 2016, from http://www.scientificamerican.com/article/reading-paper-screens/
Goodman, Elizabeth; Kuniavsky, Mike; Moed, Andrea (2012–09–01). Chapter 11. Observing the User Experience: A Practitioner’s Guide to User Research (Kindle Locations 4818–4819). Elsevier Science.
Herrman, J. (2016, April 17). Media Websites Battle Faltering Ad Revenue and Traffic. Retrieved May 02, 2016, from http://www.nytimes.com/2016/04/18/business/media-websites-battle-falteringad-revenue-and-traffic.html smid=tw-nytimes
Holst, C. (2010, November 01). Readability: The Optimal Line Length. Retrieved May 21, 2016, from http://baymard.com/blog/line-length-readability
Krug, S. (2006). Don’t make me think!: A common sense approach to Web usability. Chapter 2. Berkeley, Calif: New Riders Pub.
Marjoo, F. (2013, June 6). You Won’t Finish This Article: Why people online don’t read to the end. Retrieved March 31, 2016, from http://www.slate.com/articles/technology/technology/2013/06/how_people_read_online_why_you_won_t_finish_this_article.single.html
Nielsen, J. (2003, November 10). The Ten Most Violated Homepage Design Guidelines. Retrieved May 09, 2016, from https://www.nngroup.com/articles/most-violated-homepage-guidelines/
Nielsen, J. (1995, January 1). 10 Heuristics for User Interface Design: Article by Jakob Nielsen. Retrieved April 01, 2016, from https://www.nngroup.com/articles/ten-usability-heuristics/
Nielsen, J. (1995, January 1). How to Conduct a Heuristic Evaluation. Retrieved April 2, 2016, from Nielsen Norman Group, https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/
Nielsen, J. (2008, May 6). How Little Do Users Read? Retrieved March 30, 2016, from https://www.nngroup.com/articles/how-little-do-users-read/
Nielsen, J. (2007, July 9). Write Articles, Not Blog Postings. Retrieved March 30, 2016, from https://www.nngroup.com/articles/write-articles-not-blogs/
Nell, V. (1988). The Psychology of Reading for Pleasure: Needs and Gratifications. Reading Research Quarterly, 23(1), winter, 6–50.
O’Hara, K. (1997). A Comparison of Reading Paper and On-Line Documents. Retrieved July 26, 2016, from http://www.sigchi.org/chi97/proceedings/paper/koh.htm
Sharon, Tomer (2012–03–21). It’s Our Research: Getting Stakeholder Buy-in for User Experience Research Projects (p. 97). Elsevier Science. Chapter 3.
Travis, D. (2016, April 12). 247 web usability guidelines. Retrieved May 01, 2016, from http://www.userfocus.co.uk/resources/guidelines.html
uk.gov, Government Service Manual. (2014, June 4) Expert reviews: Getting input into products and services. Retrieved April 1, 2016, from https://www.gov.uk/service-manual/user-centred-design/user-research/expert-review.html
[button text=”View the entire report” url=”https://paulmirocha.design/wp-content/uploads/2017/12/UR-final-report_mirocha_v2.pdf” target=”_blank” color=”#b94b11″ size=”big” icon=”doc-text”]