In the era of COVID, public interest organizations from legal aid societies to public health departments have never been busier while many students have never had more free time. So, too, our civic need for effective social problem-solving has never been greater. What better time to launch a curated platform for social problem-solving and civic technology?
What if there was an online exchange and information-sharing forum, the Public Interest Project Hub (PIP Hub), that would enable students and public interest organizations to connect to share ideas and coordinate projects to address civic needs.
Public interest organizations or citizens with problems who understand public and organizational needs could add project proposals to the Hub. These ideas and proposals might range from a client intake system that facilitates real-time statistical analysis to a know-your-rights app that helps people understand regional laws and advocate for themselves, or a hundred other creative ways to address or mitigate collective problems. Students with relevant policy, design, and technological skills could then connect with the organization in a volunteer or contracted capacity to develop a project plan that leads to a research report, tech tool, website, or another outcome that helps scope or address the issue at hand.
PIP Hub, diagrammed
GitHub has been wildly successful in allowing people from across the world to collaborate on technical projects such as Bootstrap and JQuery; but, without guaranteed and actively managed student and community/organizational participants, similar spaces for public interest technology have not emerged. This is a shame as there are likely many front-line organizations who have unmet technical needs and many students who would be excited to support them in developing solutions.
Think of PIP Hub as a forum combining features from Google Drive and GitHub. The drive would be the external-facing system that students and organizations could use to connect with each other. It would contain a project idea intake form and a Google sheet listing open projects that students could either apply to or just start working on. It could also contain white papers describing organizational and community needs; completed student research on public policy and/or social problem-solving issues; and folders that link to completed or ongoing social problem projects on GitHub. The PIP Hub would facilitate the collaborative development of technical projects. It would also make it easy to build open-source projects available to a broad community of developers and users.
Students who have worked on policy/civic tech projects but who have since stepped away from their work could also benefit from the hub. Many project-based courses require students to research and prototype projects, but after the term ends most of those projects never move forward. The next year another group of bright-eyed students enters the same course only to repeat the cycle. Students who worked on policy proposals or civic tech projects could leave their projects in the drive and then allow students who take a similar course, or are just interested in the project, to pick up where they left off and move the project further toward real-world use. Public interest organizations could also view these projects, provide feedback, and write proposals for a group of students to build a tool based on a student prototype.
Some public interest organizations may hesitate to engage with PIP Hub, pointing out their need to own the data they create and control the technology they rely on. Yet, the counterpoint is a pernicious trend in civic technology with proprietary software that privatizes public data, prevents community members from understanding a technology’s impact, and creates barriers for widespread adoption. To ensure ethical and effective design, all work-products that emerge from the hub should be open source or licensed through Creative Commons. If necessary for their mission, public interest organizations should be able to request that work products be closed source.
In sum, the PIP Hub could facilitate an innovation ecosystem grounded in civic engagement that would connect public interest organizations with unmet needs to students who are seeking ways to develop their skills in policy research and/or civic technology. It could also connect students who have started policy/civic tech projects to others looking to carry them forward.
To ensure a successful innovation ecosystem, PIP Hub staff would need to perform several administrative and constructive functions. They would need to publicize and solicit engagement for the drive from public service organizations, students, and professors. They could also structure the terms on which the work would be done and facilitate project sustainability by providing students with either course credit or grant funding. Once projects are completed, the Hub could publicize the products to similar organizations that could benefit from their use. Staff could also connect students and organizations to professors or foundations who could advise or fund projects. Lastly, they could organize and index documents so that content is easily searchable. On the Google drive, this could mean imposing a standard format on the tracking sheet and grouping projects displayed for external observation by issue area to be most accessible for partner organizations. On Github this could mean managing permissions and ensuring that each project has a README file containing a comprehensive summary of the project. Financing for the PIP Hub could be provided by such groups as the Public Interest Technology University Network or the philanthropies behind it. University programs and departments could pay small dues to give their students access, thereby enabling cross-university partnerships and team experiences for their students. Public interest organizations could join as members. Corporations could sponsor the hub as donors. Throughout, the Hub would retain its independence and neutrality to enable civic organizations, community members, and students to partner freely on projects. It may also be advantageous to pilot several different hubs with different funding and administrative models based out of different research universities. After a trial period, the hubs could compare, adopt best practices, or merge into a unified system.
In short, the PIP Hub would contribute to a world where communities can join together in collective problem solving to find sustainable solutions to public problems. It would facilitate the distributed creation of high-quality research and technical tools that public interest organizations could use at low-no cost. It could help communities share information and find collective solutions to problems they identify. It could serve as a training ground for a generation of students interested in applying analytical and technical methods to societal problems.
Michael Swerdlow is a recent Stanford graduate and admit to Columbia Law School. If any organizations are interested in creating their own PIP Hub feel free to reach out. Contact: mswerd@stanford.edu
This piece documents an intervention aimed at improving access to justice outreach. We used a technical intervention — standardized data markup on legal help websites — to improve people’s discovery of key, public-interest legal information.
Intended Goal: Our goal was to help more people find out key legal rights & services when they searched online for help. Search engines’ results pages are a key place where people find out about legal help. We wanted to raise the placement of local, public interest legal help sources on Google search results pages. Ideally, a person searching with a problem like ‘eviction help’ or ‘how to get a restraining order’ will see local legal aid groups that can help them resolve their problem, or legal guides from their jurisdiction that can build their legal capability
Our Intervention: We used website structured data markup, called Schema.org, to improve how search engines found & displayed legal aid websites to people searching online.
Our Team: Our team at Stanford Legal Design Lab partnered with local legal aid groups across the US, with support from the Pew Charitable Trusts & the Legal Services Corporation.
Schema Markup Intervention
In the early months of 2020, our team worked with stakeholders in Florida, Hawaii, New York, Alaska, and Idaho to implement an intervention that could improve the placement of legal aid and legal help websites in search results. We created schema markup for our stakeholder organizations’ websites, with the expectation that this structured data could help their public interest organizations show up higher on search result pages, and could match them with more appropriate users of search.
The non-profit Schema.org consortium of major web search providers, including Google, Bing, Yahoo, and Yandex, sets the standards for the markup, and also uses it with its search engines. Schema markup is a way that websites can communicate to search engine crawlers and bots. It is JSON code that is put on the back end of an individual website page, which indicates information about the organization that runs the page as well as the services and content that is on the page itself. It uses standard markup language, that all of the major search engine providers agree on, to communicate these key pieces of information to the search engines automatically.
Markup is often used by commercial businesses in their search engine optimization (SEO) strategies. In addition to other SEO techniques, like having more websites linked to your website, creating content that matches with common search terms, and having fast loading and mobile responsive pages, schema markup and other structured data techniques can help an organization get more prominence in people’s search results.
One of the main issues with Schema markup is the high barrier to knowing how to use it for a particular type of organization. The markup itself is a collection of terms that can be defined however any given webmaster or content developer wishes to. It is free to use, but takes a great deal of expertise to understand how to use it correctly for a certain type of organization, website, and set of services.
Steering group of stakeholders, on legal help markup protocols
For this reason, our stakeholder group focused on developing a standard protocol for legal help organizations who wanted to mark up their sites with schema markup. Over the course of six months, our lab team drafted and revised a standard way to use the markup for legal aid organizations, statewide legal help portals, and other legal information websites. We did this with constant feedback from these organizations’ webmasters, lawyers, and content developers. We held twice-monthly phone calls and Slack office hours, in which we presented our proposed markup protocol for these stakeholders’ revisions and approval. Their input helped us to understand:
What types of organizational details, services, and site content they wanted to make sure that people searching online should find, and what information they would rather not be shown publicly (or shared widely).
What the right ‘unit’ for sections of markup should be: like describing all services around the main organization, or describing the services that each office of the organization provides, or describing the services that each unit of the organization provides. We ended up with a ‘graph’ model of the legal schema: describing the main organization, then describing each service it provides, and linking them together using key ids that made it clear that the organization provided the particular services.
How to actually implement the schema on their websites and content management systems, to get the JSON code onto their homepages without disrupting their current codebase and SEO strategies.
Our markup covered the pieces of information about the organization, as well as its general legal aid or legal help services. In our first round of markup, we decided not to mark up events, one time services, or other pieces of information that were likely to change quickly. Our stakeholders warned us that they often don’t take down stale information on clinics or events, so we may inadvertently display events to users that no longer take place (like bi-weekly office hours on eviction help, that may have been halted several months ago). They explained that their events are often funding-contingent, so when a 12- or 18-month grant ends, the event service ends but they do not necessarily update the website with this information.
Instead of events or other information that changes frequently, we focused on the stable information about the website, like the organizational details, contact information, jurisdiction, areas of issue expertise, price range, and type of organization. This organization-level markup may have an effect on these organizations’ sites being displayed more prominently, potentially with a map, business hours, description, and contact details shown on the results page. It was not page-level markup, that would be more likely to have specific sections of content, FAQs, or step-by-step how-to’s appear on the results page. This first round of schema markups is available for public review.
An example of Schema.org code for Idaho Legal Services’ website and legal aid services
By Spring 2020, we had an agreed-upon set of terms and ways of laying out schema markup for legal help sites. Our Lab handcrafted markup for our partner organizations, following the protocol. We gave the seven legal help organizations the text files of the markup for them to apply on the back end of their main home page. Because each group had different content management systems, there was wide variation across our cohort surrounding how easy or hard it was to implement the markup on their website. Typically, markup is implemented by manually inserting it into the backend code of a website, like in homepage’s header code. In some cases, legal aid groups use other search engine-related apps on their content management systems that interfere with new markup being added. We had intended that all sites would implement it at the same time so that we could have regular check-ins to see any effects or watch for effects of possible external variables like new policies on unemployment or rental housing for COVID. But there were gaps of several weeks and months between the different sites’ schema implementation.
Expected impact of the markup intervention
Our expectation was that the schema intervention would increase the traffic of jurisdiction-correct visitors to the website. There have not been controlled studies of precisely how markup may increase overall traffic, or jurisdiction-correct traffic. It was unclear whether overall traffic volume would go up. Instead, the goal was to increase the jurisdiction-specific traffic of people from the state or the region that the organization serves. That can sometimes be seen through Google Analytics, which shows the state or region from which visitors are apparently located. (Though, it is important to note, that people using various browsing tools may obscure their location or prevent tracking, for the sake of privacy). We also expected to increase the number of visitors who were interested in the subject matter of the organization, as shown by the time they spent engaging with the site rather than ‘bouncing’ (or leaving quickly, apparently because the site did not match their intent). We wanted to find people whose search queries’ intent matched the services and information on the websites with the markup.
This first round of the schema intervention was also meant to be a learning experience in developing the intervention itself. As there is no official protocol on how to use the myriad schema markup terms to represent a given group — particularly a non-profit group (as most markup so far has been done by commercial groups, and examples are geared towards businesses) — this experiment was meant to to identify bugs, develop a common protocol, and begin to see how much of an effect on traffic structured data markup could have. As we created and implemented the markup, the goal was to learn 3 main things:
How to automate the creation of markup so that more organizations could do it quickly if not automatically.
How best to do more detailed markup, that would even better specify key terms about service, issue expertise, and jurisdiction, to improve how search engines showed information about these sites. In the first round, we kept a fairly short list of general, site-wide information to mark up, but in future rounds we hope to mark up more of the websites’ individual pages, guides, hotline, clinics, and other services they offer.
Whether the hype around schema markup’s ability to improve site’s rankings translated into real impact. Could markup increase how often sites appeared in the top 10 or top 3 search results? Could it increase the number of jurisdiction-correct visitors, who would spend time on the site to make use of its resources? If it did not have substantial impact, this would change our strategies, to possibly focus on more engagement with policy leaders at technology companies, about how to improve public interest organizations placement, aside from structured data.
Once the seven organizations applied schema markup to their websites, we then used a combination of individual site analytics and post-markup search engine audit to analyze these three areas of inquiry. The search engine audit helps us to see what people are being shown by search engines, when they search for common legal problem terms. The site analytics help us to see who is finding their way to public interest legal help sites, what queries or referrals brought them to the sites, and how they behave when they are on the sites.
Website Analytics & Search Audit: What was different, what did the intervention show?
After implementing the Schema markup intervention on the seven websites, our team led an evaluation of what the sites’ Google Analytics showed in terms of impact before and after the markup. This is in addition to a post-markup audit of search engine results, to see if the public interest sites appeared more frequently or in higher positions after they had markup on their site.
Whereas the search engine audit informs us about what sites people are shown as likely relevant to their question, website analytics inform us about how many people end up coming to a website to find help for their problems. We use both measures to see how markup affects how search engines display listings of sites, and how users ultimately behave in choosing sites to visit and spending time on them.
Google Analytics provides an overview of website traffic. It includes reports and analytics on traffic sources, locations, demographics and behavior of the audience. We used the Google Analytics data to analyze the effects of the Schema markup intervention, and the impact of COVID-19 on the aforementioned markup intervention.
How soon does markup make a difference?
Generally, the effects of schema markup cannot immediately be evaluated after they’ve been implemented on a website. There are varying online reports, and no clear answers, but according to most informal message board threads, the effects of markups are visible on search engines and analytics pages anywhere between 1 week and 1 month. It may take a while because of the cycles on which search engines’ crawler bots search the Internet to index new sites, content, and markup. They may be delayed in ‘finding’ the markup, if they only visit the site once a week or once a month to look for updates.
In our project, the schema markups for the legal aid and service organizations were all implemented on their websites on different dates. In order to provide a valid evaluation, we had to ensure that enough time had lapsed between the implementation of the markup and the analysis.
Schema Implementation Records
Organization & Schema markup implementation date
Idaho Legal Aid: 5/6/2020
Legal Services of North Florida: 5/11/2020
Law Help New York: 5/29/2020
Legal Aid Hawaii: 6/2/2020
The above info provides an overview of the implementation dates. The analysis was conducted on the websites’ analytics at the end of August 2020.
How are the sites performing?
Two main sources were mined for data: Google Analytics and Google Search Console. It is important to distinguish these two tools. Google Analytics provides more insight about the visitors, page visits, and usage time of the webpage. It tracks and reports different segments of website traffic. Google Search Console, on the other hand, provides more insight on the organic traffic search results. Organic search traffic indicates the visitors that visit the website through search engines. We used both of these data sources to analyze the impact of the schema markups.
Measurement issues
Before delving into the results there are some measurement issues and important assumptions that may have impacted our results in visible ways. These issues will be discussed more fully in the following paragraphs.
Different implementation dates: The implementation date is different for each of the legal aid and service websites. Although we left the minimum required time (1 week) between each implementation and the start of the analysis, there are some unknown aspects regarding the detection of the markup by search engines. This might have created some variances in the analysis.
Missing data: There was the issue of missing data for some of the legal aid websites. Florida Law Help switched ownership in early 2020. Google Analytics was disabled when it was moved to a different platform, therefore there is missing data in the months of April, May and June 2020.
Correlation/Causation: Although some of the data suggests that there is a possible correlation between the schema markup and the several of the metrics we analyzed, this does not necessarily indicate causation.
Results: Did Schema improve visitors and quality matches?
We looked at four separate metrics to determine if and how schema markup changed the legal help websites’ discoverability: visitor count, traffic sources, session duration, and click-through rate. We recognize that particularly in a year with COVID-related upheavals, people may have changed their search behavior, and that this could confound our reliance on analytics. Ideally, in future years with fewer emergency events, there can be further study of website analytics before and after the implementation of markup or other SEO strategies.
Metric 1: Visitor Count
When thinking about impact, increasing the number of visitors to a website may seem to be the most important metric. Yet we know from our discussions with legal aid experts that they are most interested in increasing the number of ‘appropriate’ visitors — those from their service area and who have legal help queries — even if that means a decrease in overall visitors.
Visitor count can also be a problematic metric for measuring the effectiveness of a particular intervention, like schema markup. The visitor count metric will be affected by other variables aside from markup — like policy changes, economic downturns, natural disasters, and other events that may affect who is searching for legal issues. We can expect that a large swing in visitor count is likely due to an annual or special legal event. Theoretically, comparing the visitor count trend for 2020 with the trend for 2019 can help separate out the seasonal changes in visitor count from the effect of schema implementation, but because of the unique legal circumstances in 2020 (i.e. COVID lockdown, economic downturn, and related legal issues around housing, benefits, unemployment, family, schooling, etc.), such a direct comparison was not possible either.
Therefore we analyzed 3 other metrics, in addition to visitor count: traffic sources, session duration, and click-through rate.
Metric 2: Traffic Sources
We compared where sites were getting user traffic from. In particular, we compared direct traffic versus search traffic. Direct traffic means users are arriving at the website directly through a link or a bookmark, while search traffic means users are arriving through searching through Google. If an increase in visitors was caused by schema, we would see an increase in search traffic only, but if the increase was caused by current events then it is likely that both direct and search traffic will increase.
As the column chart indicates, there is a spike in visitor count for search traffic in the month of July after schema was implemented on 5/29/2020. This means that there was an increase in visitors who visited the website through organic sources, such as search engines. A possible explanation for this spike could be that rich results and better ranking on search engine result pages generated by schema might have established an environment that produced more clicks in the month of July.
Figure 1: For NY Law Helpcolumn chart depicting direct and search traffic per month for the year 2020
Metric 3: Session duration
The metric of ‘session duration’ is significant for our analysis, as an indicator of a good match between the site’s content and the user’s intent. The schema implementation aims to increase not just the volume of search traffic to legal help websites, but good matches to the person’s actual needs. If there is an increase in session duration, then this might suggest that the website is attracting more people who find value in the website’s information and are willing to spend more time engaging with it. The visitor session duration metric can also tell us if an increase in visitor count is due to the website being discovered by people who really need legal help from that state, or if it is due to the website being suggested to people who click on it but then realize it is irrelevant to their situation.
For Idaho Legal Aid, we saw a very drastic increase in page views per session for visitors who were on the website for 30+ minutes. This might suggest that the website is attracting more people to whom the website’s content is relevant.
At first glance, it seemed that Legal Aid Hawaii also enjoyed an increase in visitor count, but it turned out that the gain was due to an increase of visitors who stayed for 0–10 seconds. These visitors are more commonly known as bouncers. Law Help NY also saw an increase in bouncers, but there was also a slight increase in the number of visitors who visited for 10 seconds — 10 minutes.
Figure 2: The Idaho Legal Aid website saw an increase in pageview per session after the schema markup implementation.
Metric 4: Click-through rate (CTR) and search ranking
The last metric that can be useful is the click-through rate and search ranking. Indeed, this might be the most appropriate metric to evaluate the markup’s effectiveness, since Google communications claim that schema markup will improve search ranking. Of course, whether that improved search ranking means that more people will click on the search result and interact with the website is a step removed from improved ranking. Florida Law Help had incorrectly implemented their Google Analytics, but they had provided more extensive Search Console data than the other websites. The click-through rate generally increased after schema implementation. Search rank was on the decline until the markup was implemented — then, the website rank steadily started to improve.
Figures 3 and 4: click-through rate and search ranking improvement for Florida Law Help after the schema markup implementation.
Searchers for the organization versus the issue
Two good indicators for whether schema would be effective for a given site are: 1) the proportion of search traffic to direct traffic and 2) the proportion of clicks after the user searches the exact name of the organization. We chose these indicators because markup would only affect a person’s online behavior if they were searching help through a search engine instead of going to the website directly. Also, markup-driven increased search placement or rich snippets are more likely to affect a user’s chances of clicking on a website if they were looking for general legal advice. Users who were already looking for a specific organization’s website will look for and click on the organization’s website regardless of whether there’s a rich snippet or higher placement.
Around 50% of Legal Services of Northern Florida’s traffic was direct, which means that most visitors of their website never got to see the rich snippets at all (since schema implementation adds rich snippets to search results). For both LSNF and Hawaii, 34% and 18% of visitors respectively had typed in the exact name of the organization in the search bar. This suggests that a good portion of visitors to these sites were already looking for the organization’s website, so a rich snippet or other special search engine treatment would not increase their chances of clicking on their results.
Comparatively, only 5% of Idaho’s visitors had typed in the exact name of the organization. Since most of Idaho’s visitors are people searching more generic help terms (like “tenant eviction help idaho,” instead of “idaho legal aid”), the rich snippets can really help draw attention to the site. A legal aid group that wants to increase its search traffic could use schema markup to connect with people searching for their problem issues, rather than for a local legal aid group.
Figure 5: LawHelp Minnesota’s traffic source is mostly search traffic. This signals potential for schema markup implementation.
Confounders of pre- and post-markup search analytics
We recognize that other factors aside from the application of schema markup may have affected the websites’ analytics.
One set of confounding variables are content and technical changes on the websites we are working on. For example, session duration could have also increased due to technical changes on a website. A more complex layout, for example, often means that the website is difficult to navigate. This could be a potential factor that increases the session duration. After a check-in with the public interest groups, we found out that none of the layouts have changed since the implementation of the markup and we could therefore proceed with using the metric in our analysis.
A second set of confounders are around seasonal changes. People’s legal help situations may differ throughout a given year, rising and dipping with different seasons and regular events. Financial legal needs may spike around tax season in the early months of the year, or during the holidays when more money is being spent. Educational problems may rise before and during the first months of a school year. Search traffic may be higher in certain times of the year, and thus there might be changes in search traffic during the analysis period that is based more on changing legal needs than the markup intervention.
A more difficult set of confounding variables emerged with COVID-19. The health and economic emergencies led to both new legal protections, and more people facing legal help situations like needing to file for unemployment benefits, avoid eviction for nonpayment of rent, deal with domestic violence threats, and deal with contracts affected by pandemic restrictions. These new protections and problems likely changed people’s search behavior. We might assume that the pandemic increased the levels of searches for legal help, and also introduced new types of searches (like, ‘is there an eviction moratorium in my state’ or ‘how do i claim my stimulus check’). This changing search behavior presented a major confounding variable for our intended pre-/post-study of the markup’s effects on legal help search.
At first, we thought that we would wait out the emergency period, and run the intervention once it ended. Once we realized that the emergency would be continuing indefinitely, we considered how to incorporate the data fluctuations into our analysis. We adjusted our evaluation, to look not only at pre- and post-markup metrics (both in the year 2020), but also comparing the same time period to the previous year. This method allowed us to spot fluctuations that were likely caused through changed search behavior relating to the pandemic, as opposed to seasonal confounding variables that would occur each year. Moreover, as we noted above, we decided to use multiple website analytics metrics in addition to the visitor count, which would be more susceptible to fluctuations due to current events, such as Covid-19.
We tried to use these techniques to minimize the effects of Covid-19 on our data analysis. But there is no guarantee that we managed to fully eliminate the impact that Covid-19 had on the data analysis, and we recommend that future studies in a more ‘stable’ year will be needed to measure markup’s impact.
Search Engine Results Page Audit, round 2
After implementing the schema markups on our partners’ sites, we conducted a second round of a SERP audit. This second audit was to determine if, after the markup implementation, we saw any differences in how public interest legal aid sites’ placements. We compared the first SERP audit’s results with the second’s, to see if overall three was any change in domain types, and for specific websites’ performance.
One change we observed was a decrease in the amount of .org domains in Northern Florida-based searches.
We had put markup for two legal help websites with .org domains that try to serve users with legal help queries: https://www.lsnf.org/ ; https://floridalawhelp.org/. Our assumed outcome is that these .org sites would appear more frequently based on the markup, intervention but this did not occur. Interestingly, there was an increase in .gov links that appeared in the results. This was most likely not due to our schema intervention, though. Our markup was only added to organizations at .org domains, and not with .gov domains. The increase in .gov links could be explained by the Covid-19 information and help pages set up by governmental institutions. Their improved website offerings may have increased their search placement.
Hawaii’s second SERP audit illustrates a similar trend. There is a slight increase in .gov domain extensions, but as noted above, the stakeholders that participated in this research project did not have a .gov domain extension. The slight increase in both .gov and .com domains can potentially be explained by the Covid-19 pandemic that created a surge in both searches as well as the creation of help and information pages regarding the pandemic.
Figure 7: Percentage of non-US domain names
There also hasn’t been a significant change in the percentage of non-US domain names. This percentage seems to be nearly the same in November as in January of this year.
The lack of development and changes in the search audits pre- and post-schema markups may be explained by two factors. The first factor is the limited duration of the schema implementation on the websites. As noted earlier in this chapter, most stakeholders implemented the markup in the early summer months. There is no clear explanation from search companies or research data on how long it takes before the effects of the markup can be seen on the result pages of search engines. It might take several more months before we can see the markup being recognized and understood by the search engine crawlers and subsequent ranking algorithms. The markup merely speaks to any crawlers about what is on the page, but there is no documented, predictable process about how the search engine teams and algorithms crawlers take the markup information and use it to affect the search results. There is also the possibility of advocacy to technology companies, to have their search teams pay particular attention to this markup so that they track and evaluate internally how well they are serving legal help searchers.
The second factor is more technical. The schema markups have mainly been implemented on the mainpage of the organizations. However, it might be possible that the benefits of schema are mostly gained when the markup is implemented on the specific pages of the organization’s website. For example, describing that Legal Services of Northern Florida is a legal aid organization that serves the Pensacola and Jacksonville jurisdictions on the main page may not be sufficient. Our first round of markup interventions, focused on organization markup, may not be nearly as effective as issue- and service-oriented markup. Future interventions and analysis should concentrate on adding markup to their specific hotline, guides, and other content on specific issues — like for unemployment, veterans, landlord-tenant, or domestic violence. The main page often embeds general information about the organization, but that’s not necessarily what a user needs. Users are often looking for information on a specific problem and are thus more likely to click on a link on a search engine results page that provides a snippet with specific information. They would be searching for their problem, and not for ‘legal aid group near me’. This means that in future iterations of this project, more attention has to be given to schema markups for specific legal information pages.
Did the Schema.org intervention change the legal help sites’ performance?
Because of the confounders, particularly around COVID-19’s effects on search behavior, we cannot make strong conclusions about schema markup’s effects on traffic to public interest sites. We can use the pre- and post-markup analytics to observe some changes that indicate for some sites experienced higher numbers of visitors and improved click-through rates. But not all sites experienced these increases. This 2020 analysis suggests that there might be value in schema markup to increase search rank, but marking up at the organizational level does not produce a ‘markup bump’.
Speaking with our legal aid practitioners and search engine contacts about these results, some alternate proposals emerged. In future interventions and evaluations, there might be a higher increase in traffic if more of the sites’ individual webpages and services are marked up (not just the general organization). In this way, their specific guides, articles, FAQs, hotlines, and clinics on issue areas like renting, debt, domestic violence, etc. may be communicated to search engine bots. These bots can recognize that these individual pages have content that can help people searching for problems in this area, and then do a better job in matching searchers to this specific page (not the homepage) that can help them with their query.
The results indicate that there might be value for some sites in using markup to increase traffic, it is important to look at the schema markups in more relative terms. Is this intervention, of developing and applying structured data on a website, meaningful enough for wide application in the public interest legal help community? How much does the schema markup improve the search engine results and increase its amount of targeted visitors?
Exact understanding of markup’s impact is difficult to define. Google’s search team and its algorithm do not list conclusively how their algorithm uses schema markup in its search results, and how this differs for particular areas of questions (like for legal help). Even though Google encourages sites to use markup to improve their search placement, there are many other search engine optimization techniques that sites may use, such as mobile-responsiveness of the websites, user-friendly content, search experience of individual users, page-loading times, preventing spam on webpages, etc. Even if a legal aid site has markup, other sites who are using other SEO techniques may still place above them. Implementing schema may increase the likelihood that a site’s content may appear in rich Google search results or better targeting of results to user’s queries, but does not guarantee either.
In the future, Google’s search engine may treat the sites differently if the markup specifies that they are authoritative — with a government designation, public interest credentials, or another standard that is set as authoritative. This would depend on the legal community’s ability to decide what a marker of authority could be, and then discussions with search companies to make them aware of these markers and why they should be considered in their algorithm. For example, a coalition of courts and legal aid groups could identify what makes their sites more authoritative and beneficial to people seeking legal help. They could agree on how to use schema markup terms to communicate their public interest status. Then they could convey this work to the search companies, and attempt to inform them about how they are using markup to designate their authority and why the search algorithm should pay regard to this part of the markup.
Image: An annotated version of our schema markup, that was revised after our initial implementation, to better allow for building onto it with individual pages + articles marked up as well as the main organization and legal aid services.
When we are discussing future work and next steps, it is important to realize that the schema markups are just one part of a larger set of SEO techniques and search engine policy decisions. Markup will not be the sole solution to improve search results, create rich snippets and other special treatment on the results page, and drive more traffic to public interest legal help sites. Equal attention has to be given to creating user-friendly content, easily accessible web layouts, fast loading webpages, compatibility with various devices and browsers, etc. Last but not least, another important factor is setting up partnerships with tech companies. These companies can not only provide public interest groups with the correct information on how to adjust their websites and markups, but there is opportunity for more conversation on why there is a need for legal help snippets and better search engine page results. It is only through a combined effort by both parties that we can increase the accessibility of online legal information.
A big thank you goes out to our research assistants, Yue Li and Julia Park, and all of the wonderful pioneering stakeholders who participated in this research project: thank you!
set up topic pages and create an overall site architecture on a statewide legal help portal
title pages so they’re easy to find
phrase complex legal concepts, so they’re plain language & accessible
staffing and managing the development of legal help content
integrating forms and other official content into your site
This guide can be useful to your team that’s working on government websites, public interest guides, legal aid websites, and court self-help tools.
Come explore and get more guidance on making an excellent, user-friendly legal help site with relevant content, great SEO, and usable text for people seeking help.
Are you a court or legal aid group that’s hoping to serve more of your audience with the website you’ve built?
Dave Guarino, a civic technologist, has written an SEO Cookbook that introduces non-technical legal professionals to the essentials of Internet Search & how to optimize your website to place higher when people search for their legal problems.
Our site presents Dave’s SEO Cookbook for you to read & use as you make plans for improving your website & its rank.
Please let us know if you have additional questions or needs, that we can help you solve.
Most courts and legal aid groups have websites to help the public. They have rights, forms, guides, contact forms, and intake sign-ups to help people get help in a crisis.
But are these websites set up to be as discoverable, usable, and effective as they can be?
Based on our review of the key setup & features of a website, we have focused on 4 key areas for a well-designed website: discovery, technology, content, and interface design. We’ve been reviewing and ranking sites on how they do in these 4 areas.
We have taken one step back from these detailed rankings & seen an overall model emerging. We’re ranking regions around the country regarding their website coverage & performance.
Where are they on this 5-option maturity model?
The key indicators and decisions are about those 4 areas. But we find that many websites that have problems in one area (like not having plain language content) also have problems in other areas (like outdated technology and poor discoverability).
That is why we are now grouping websites into this overall maturity model, to help regional leaders see if their local legal help websites are mature enough to serve the public as they’re intended to.
For those of you who have missed the news, OpenAI trained a model (ChatGPT) that answers queries in a conversational manner: it responds to follow-up questions, corrects mistakes and challenges/rejects inappropriate requests.
Is AI Chat the New Google Search?
I found people raving about this new chat model. One particular tweet really stood out me:
ChatGPT seems like such an amazing new way to search for information online. I was eager to test it out for myself, in particular for one of my favorite research interests: online legal information.
What would AI Chat Mean for Legal Help Info?
As you all know by now, one of our aims at the Lab is to connect users to jurisdiction and legal issue-specific information that is up-to-date and freely accessible (so not hidden behind paywalls). We are working towards a reality where Google and other search engines provide snippets and legal knowledge panels, similar to what they do with health knowledge panels. As most people search for legal information online, providing users with jurisdiction and issue-specific information would ensure that we increase legal capability, access to justice and that we empower users during their legal journey.
Image rendering by Margaret Hagan
Legal Search Results Are Often Low Quality
Compared to that vision of a clear, authoritative search results page, we are still not there.
The top results on search engine result pages are not jurisdiction-specific. They direct users to websites with most of their relevant content behind paywalls. Or, most often, search engines send people to websites with content-farmed generic information that is technically ‘correct’ but not actionable. These generic short articles don’t provide the user with good quality legal information that they need to proceed in their legal journey.
In the worst-case scenario, search engine results can be outright misleading. Take this results page for a free counsel query, that’s searching for a free lawyer for an eviction lawsuit.
A person facing eviction in San Francisco is guaranteed a free lawyer under the new “Right to Counsel” as of Summer 2019. But a Google Search for “free lawyer for eviction SF” directs them to a Knowledge panel from a commercial law firm.
What do search engines show to landlords versus tenants?
I have also recently observed that there is not an equal playing field when it comes to search queries from tenants and landlords.
Compare these two images and queries. One is from the perspective of a tenant, looking for information about eviction. It has no snippets or knowledge panels and the second and third result on the page are websites with generic information. The other image is a query from the perspective of a landlord. The landlord receives a legal snippet from the California Court’s website.
These user-friendly snippets should be available for all parties to ensure there is an equal playing field. Why is the search engine showing a clear checklist to one party, but not another?
So far, I have not heard a single argument why this could not be possible.
Can AI Chat do better than Google Search?
You can therefore understand my excitement when I saw the ChatGPT results. What would happen if I asked the chat model my usual eviction queries?
The information provided by ChatGPT is correct. I also applaud the plain language. Unfortunately, the information is too generic to really increase legal capability and empower users on their legal journey. So let’s see what happens when I ask a follow-up question:
Although the bullet point list is excellent and easy to digest, the information that is provided is too generic to really make a user understand the next steps.
Obviously, at this point, I did not input a jurisdiction in my query. Most users would not mention their location in a query and I was secretly hoping that ChatGPT would prompt me to do so. It did not and I definitely think there is more ground to be gained on this front.
Again, the information is not wrong, but I was just fervently hoping that ChatGPT would actually mention my rights as a tenant in Palo Alto.
I was also curious to see if there would be differences in how queries from landlords would be treated, so I tried out some queries.
My first query was without and the follow-up question was with a location. Queries without location are too generic to be truly useful for users searching for online legal information. The queries with location for landlords are slightly less generic than for tenants. Again, I wonder where this discrepancy stems from? Both tenants as well as landlords need access to good quality legal information. There needs to be an equal playing field for both parties.
ChatGPT could gain so much ground by delving into the world of legal information. Even small tweaks such as prompting users to input their location if they ask a legal query would already be a massive win. It would push access to justice into another realm if, in the future, we could have ChatGPT walk users through the process of their legal query and provide specific, high-quality legal information.
There is a High-Quality Supply of Legal Info Online. It’s Just not being shown by platforms.
Jurisdiction and legal issue-specific, high-quality legal information already exists online. Courts, legal aid organizations, and others have been working tirelessly to create content for users with a legal query.
Unfortunately, these pages cannot compete with commercial websites in search engine page rankings. This is a huge loss for everyone involved, from the legal community to the tech companies.
Search engine pages are the starting point of people’s legal journeys. Millions of people use the Internet before they consult legal professionals. If search engine pages make the first step on this legal journey haphazard or outright dangerous in case of misleading information, it not only affects an individual’s legal journey but also erodes trust in the legal system in the long-term.
Legal Help Groups Need To Work with Online Platforms
Legal professionals, researchers, and tech companies need to actively work together to make sure everyone can access good quality online legal information. Please do not hesitate to reach out if you want to collaborate on this topic:
P.S. What About the Legal Images?
By the way, as for DALL-E (the AI image generator), I had hoped that DALL-E would be a way for legal aid organizations and courts to stop using stock images.
User research indicates that stock images come across as impersonal, especially when one is searching for information about stressful and traumatic legal issues.
At the Lab we always advise organizations to put time and effort into creating customized imagery. After playing around with DALL-E and eviction-related keywords, I’m going to hold off on that recommendation for the time being, as I do not think that melting AI faces would help make legal information websites feel more welcoming.
This is what AI comes up with for eviction imagery.
Many legal aid and court groups build their website with the assumption that people will start at the home page. But that might not reflect people’s actual journeys through your website.
Where do people start on your website?
If most of your visitors are coming from Google Search, they are likely being sent to where your detailed content is. Google Search will try to match their search query with the page and content that has the substance that will help them. That means Google Search may be sending them to your pages with FAQs, guides, tutorials, and other areas with high amounts of specific, detailed substance.
Unless a person is searching for your organization by name, Google Search is likely not sending them to your home page.
So what does that mean for your website design? Of course, still invest in a great homepage with clear messaging, design, and support. But on your sub-pages, especially those with lots of detailed content, then be sure to have links, navigation tools, and other important resources there on those high-traffic, detailed pages. For example:
Putting other common documents and pages that people with this problem might have Google Search may have put them in the deep-end. Could you give them more context and links, that might give them other context and tools.
Adding in an off-ramp to Legal Aid phone numbers or intake processes
Emphasizing the jurisdiction of the site, so that the person coming to this page knows if it applies to them or not.
Signaling the authority of the site so that the user knows that it is trustworthy, free, and authoritative.
Even when people are starting at the “end” — by going straight to the detailed resource end-point rather than the starting home page — your design can support them wherever they land.
What should the landing page of your legal help website look like? What should it say, and what’s its purpose?
Our team has been working with courts and legal aid groups to design websites for the public. From this experience, and reviewing other groups’ websites, we have a few guiding principles for putting together a good landing page for a legal help website — especially one that is serving a variety of different legal issues.
Here is a discussion of the design choices we made on the Virginia Legal Aid Guides website, to discuss the important design/legal principles we were following to help our users.
This video walks through design principles for a good Legal Help webpage design. It covers:
Flagging Jurisdiction frequently, to communicate which people the site is for — and who it’s not for. We use frequent references to the state jurisdiction that this site is for.
Signal reliability and free accessibility by highlighting the common, known concept of “Legal Aid” and references to the trustworthy group that runs it — in this case Legal Services of Northern Virginia
Providing a Bird’s Eye View of the scenarios that the website can help a person with. Many people won’t know whether their life problem will be covered in this website. Give the main categories of scenarios your site can help with, plus some of the most common specific scenarios people might be searching for help with
Give Off-Ramps to human services (with phone numbers to call, offices to visit, online intake to fill in) for people who don’t want to be using online guides. Also give off-ramps to other jurisdictions, so they can find help in other places.
We have had many requests from our colleagues on the Legal Help Online Cohort, about how to get schema markup onto their Drupal website.
Here is a first way to do get key Schema markup onto your homepage. We’ll share future step-by-step guides for getting page-specific markup to indicate your jurisdiction and issue area to search engines.
Putting Legal Schema onto your homepage
Make Legal Schema Markup on our free generator here, by filling in a form and creating json code. Save this code as a txt file, or have it ready to copy for step 7.
How do legal professionals know what people are searching for online? This is important, so they understand their needs, phrases, and ways to engage them.
One way to do this research is through SEO-oriented tools, called Keyword Research tools.
Answer the Public takes the 1-3 keywords entered by the user, and then combines them with various prepositions or suppositions, enters them into Google search boxes, and records the AutoComplete suggestions that Google provides.
Also Asked & People Also Ask takes a similar approach, but records what Google Search presents in ‘People Also Ask’ boxes on the results page. QuestionDB searches popular forums, like Reddit, Quora, and Stack Exchange, for any questions that have these keywords or their variations.
They list back these posts’ titles, text, and original links.
