Why Humans are Just as Important as Machines
More than 500 years after the invention of Gutenberg’s printing press, technology innovations like digitization and information retrieval represents monumental shift in the way we discover and consume the written word. The process of assigning tags to digital content, commonly known as metadata tagging, now makes it easy to search and find specific text from massive volumes of content in a matter of seconds. This process has opened the door to new revenue opportunities for publishers who can seamlessly distribute their content to a wider audience while fueling new products that would not have been feasible to create in the past.
Metadata tagging is the process of creating a term that describes a keyword or phrase and assigning those tags to the digital assets in a publication or document. The tags don’t appear to the user, but are in the source code. This helps tell search engines, browsers and other tools what the content is about and how to display the information. The term “meta” actually means its data about data. While it sounds like simple classification or indexing, the significance of metadata can’t be overstated. It’s proven to be critical for content discoverability. And, the more discoverable the content is, the more it will be downloaded, purchased, reviewed, cited, etc.; all of the actions that make publishers relevant and successful.
The better the metadata and the more accurate tagging, the better the discoverability of the content. Obviously, different publishers will want different elements tagged, but they all will require accuracy.
Not only will publishers benefit from clean, accurate metadata from a discoverability perspective, they can also create bundled packages of content (books, journals, conference proceedings, newspaper articles, video clips, etc.) to sell to specific target audiences and market sectors. Additionally, with the proper rights in place, content providers can parse data from individual sources, and create customized content sets for prospective customers.
AI and the Role of Humans-in-the-Loop
Metadata tagging has traditionally been a manual and often laborious process. It was mainly executed through a rule-based system encoded by a specific knowledge worker or subject matter expert (SME) alongside software engineers. Rule-based tagging focuses on tagging recognizable elements clearly defined within the content. For publishers, this would include pre-defined information like:
- Title
- BISAC
- Author
- Publish date
- Academic discipline
- Age range for readership
- Language
- Price
However, with the introduction of artificial intelligence (AI), metadata tags can be created at a much faster rate with better accuracy, completely automating the process which has resulted in quicker turn-around times, less reliance on resources and cost savings. For instance, AI is extremely helpful for creating critical tags for content outside of this pre-defined information and driving enhanced search and discovery. On the flip-side, rule-based solutions reach a saturation point in value delivery due to its cognitive limitations.
While it is certainly tempting to completely automate this process with AI, there is risk for error and potentially generating inaccurate tags. After all, AI is only as smart as what it is taught. The limitation of rule-based systems is that as content evolves (new titles, new taxonomy branches, new use cases), the rules must be updated via collaboration between the subject matter experts and the software engineers. This is extremely slow and limiting. Each software change creates risk, as complex or tacit knowledge topics are usually poorly encoded by the software teams.
Today, metadata tagging can be programmed by simply showing examples to computers. This shifts most of the onus of encoding knowledge from the software engineer to the subject matter experts. And this is good! More power to the experts who can break out of the rule-based box with the help of AI. Notice the word help. AI alone is not a magic bullet. Metadata tagging still requires human expertise to help improve efficiencies and accuracy. Only the subject matter experts can help teach the machine resulting in smarter and more efficient output.It’s this human-led learning that helps keep AI technology up-to-date with the changing world, without having to change complex rules.