persian

By Jessica Weiss ’05

The University of Maryland has received a nearly $300,000 grant from the National Science Foundation that will support efforts to improve the way handwritten documents from the premodern Islamicate world—primarily in Persian and Arabic—are turned into machine-readable text for use by academics or the public. 

Assistant Professor Matthew Thomas Miller and Mellon Postdoctoral Fellow Jonathan Parkes Allen, both of the Roshan Institute for Persian Studies, will work with researchers at the University of California San Diego (UCSD), led by computer scientist Taylor Berg-Kirkpatrick, on the innovative humanities-computer science collaboration. UCSD received its own $300,000 award.    

Over three years, the researchers will work in the domain of handwritten text recognition, which are methods designed to automatically read a diversity of human handwriting types with high levels of accuracy. 

“This work has the potential to remove substantial roadblocks for digital study of the premodern Islamicate written tradition and would be really transformative for future studies of these manuscripts,” Miller said. “We are very grateful to the NSF for its support.” 

This latest research proposal builds on a number of ongoing efforts to develop open-source technology to expand digital access to manuscripts and books from the premodern Islamicate world in Arabic, Persian, Ottoman Turkish and Urdu; Miller currently leads an interdisciplinary team of researchers on a $1.75 million grant from the Mellon Foundation as well as a $300,000 grant from the National Endowment for the Humanities.

There are hundreds of thousands—perhaps even millions—of premodern Islamicate books and manuscripts spanning over 1,500 years, from the 7th–19th centuries, forming perhaps the largest archive of cultural production of the premodern world. Scanning and digitization efforts over the last decade have made images of Islamicate manuscripts in a large number of collections available to the public. However, they remain mostly “locked” for digital search and manipulation because the text has not been transcribed into digital text.  

The task is made more difficult by the diversity and intricacy of many Arabic manuscripts, said Allen, who is a historian of early modern Ottoman religious and cultural history. They may be written alongside diagonal notes, annotations and corrections, in multiple colors and “hands.” 

Under the NSF grant, researchers will develop new techniques that remove the need for extensive manual—or human—labor, a method known as “unsupervised” transcription. Eventually, the tools under development will produce models that will be able to automatically transcribe large quantities of Persian and Arabic script in a multitude of different styles with substantially higher degrees of accuracy than is currently possible.

“The Arabic script tradition is so extensive and so broad,” Allen said. “People need to be able to read these manuscripts, search within them, and integrate them into their research.” 

Image: Staatsbibliothek zu Berlin, Ms. or. oct. 3759

By Jessica Weiss ’05

The Andrew W. Mellon Foundation has awarded a $100,000 grant to support the continued development of user-friendly, open-source software capable of creating digital texts from Persian and Arabic books. 

Matthew Thomas Miller, assistant professor in the Roshan Institute for Persian Studies in the School of Languages, Literatures, and Cultures, leads an interdisciplinary team of researchers from Northeastern University, Aga Khan University (AKU) in London and the Maryland Institute for Technology in the Humanities at Maryland. The Mellon Foundation has been funding the team’s work since 2019.

“We are honored that The Andrew W. Mellon Foundation has again supported our efforts,” Miller said. “They have been global leaders in building open-source tools and open-access collections for the expansion in access to and digital preservation of cultural traditions across the world, and we are delighted to be a part of these efforts.”

The project, known as “OpenITI AOCP,” aims to enable the digitization of texts from the premodern Islamicate world—an enormous tradition stretching over 1,000 years. The tools being created by the project team will be free and open to use and will allow academics and the public to produce high-quality digital transcriptions of Persian and Arabic printed texts, from poetry to the Quran. 

“Premodern Islamicate textual production is a massive and understudied archive that remains particularly underrepresented in the field of digital humanities,” Miller said. “This democratization of access to digital text production will change the landscape of Islamicate studies.”

Thus far, the project team—made up of computer science and humanities experts—has successfully improved the accuracy of Persian and Arabic optical character recognition (OCR) tools, which are tools that transfer printed text into machine-encoded text, and have begun experimenting on Ottoman Turkish and Urdu. They are integrating those tools into a platform called eScriptorium. They also held a training session at the University of Maryland in 2020 for OCR experts from all over the world. And they taught a Spring 2021 Global Classrooms course, “The Islamicate World 2.0: Studying Islamic Cultures through Computational Textual Analysis,” on the basics of computational textual analysis as it relates to textual data about the Islamicate world.

Next steps include finalizing the open-source software for widespread use, as well as holding additional workshops and community building activities around the new tools. This latest Mellon grant will last one year. 

Earlier this year, Miller was awarded $282,905 by the National Endowment for the Humanities to support the project.

Image description: The introduction to George B. Whiting's Kitab fi al-Imtina‘ ‘an Shurb al-Muskirat, published in Beirut by American Mission Press in 1838 and housed at Harvard's Houghton Library (*98Miss168). Licensed for non-commercial use.