Interview with Mārcis Pinnis, the Chief AI Officer at Tilde

In this interview, we hosted Mārcis Pinnis, the Chief AI Officer of Tilde, which is an elite European language technology company based in Lithuania.

Marcis took us in the world of machine translation in digital media, voice-to-text technologies, AI assistants, and chatbots, presenting their open-source LLM TildeOpen, which includes over 30 billion parameters and is designed for Baltic and Eastern European languages. Finally, he introduced their FFplus sub-project, where the team is developing a locally deployable enterprise assistant solution that will allow organisations to transform their documents into queryable knowledge bases, ensuring complete data security and sovereignty.

Mārcis Pinnis, the Chief AI Officer of Tilde

Copyrights: Tilde 

 

To start, could you briefly introduce Tilde?

Marcis:

Tilde is a language technology developing company. We are based in the Baltic states. We have three offices in Latvia, Lithuania and Estonia. And we developed various language technologies such as machine translation systems, speech recognition and synthesis systems and virtual assistants for a wide variety of customers, including businesses, international organizations, governments, and also individual users.

 

From your perspective, what are the currently transformative trends that are changing this area? And, how are these areas covering machine learning in digital media, voice-to-text technologies, AI systems and chatbots, actually implemented?

Marcis:

This change of generations for technologies has been happening actually since the first large language models came out, by OpenAI in 2022. But it's not a very fast process. But the main trend is that large language models are taking over various tasks that typically were done by previous generation systems and artificial intelligence solutions.  And this trend will continue. We'll see more and more tasks being automated using large language models. And the current trend is that large language models are becoming multimodal. They can process text, vision and also speech. And that will also continue. And in Europe, a big trend currently is that large language models that are being developed in Europe are trying to ensure language equity across all European languages, which is very important in the European Union. So languages are core values of the European Union as such.

 

 Tilde has been entrusted with building TildeOpen. This is an open-source, fundamental LLM with over 30 billion parameters designed for Baltic and Eastern European languages, specifically. Could you explain how generative AI is integrated into your workflows and solutions? And can you introduce some concrete examples of how your clients are using these capabilities to solve their real-world problems?

Marcis:

We are trying to integrate large language models in most of our products. For instance, in machine translation, we try to allow users to have more flexibility in terms of how to style their translations, and because large language models are more capable and have better quality that transforms into, which translates into increased productivity for customers down the line, for instance, translators have to edit less, which means cost savings typically.

Machine translation is one example. Unlike virtual assistants, current virtual assistants use large language models to enable technology called retrieval-augmented generation, where we can quickly deploy virtual assistants by just uploading documents into the system. And when users ask questions, we first find the relevant segments from the uploaded documents and then ask a large language model to answer to that user's question.

Before that, we had to define intents. And for each intent, we had to train a model that is able to recognise that intent, and it was a much more complex process. So large language models allow us to be faster in terms of deploying technologies. So these are two examples, and there are more, obviously.

 

This brings us closer to your FFplus sub-project, which you are now working on. Could you briefly introduce your use case and what challenges it addresses, and what innovative solution you have been developing?

Marcis:

In the FFplus sub-project, we are developing a locally deployable solution that allows organisations to manage knowledge. So document locally, and then make them available, accessible to their employees and maybe also to their public, if that is what they want, and transform the organisational knowledge into a queryable solution where users can ask questions and receive the answers on various topics.

This solution has multiple challenges. One challenge is how to deploy everything on a single server, for instance. And how to deploy large language models locally, and how to ensure the efficiency of the whole process. So there's this engineering aspect. And the other is that within this sub-project, we are also fine-tuning large language models and specifically TildeOpen, the foundational large language model that we have trained, for these downstream use cases that are necessary for this local solution.

And those use cases are machine translation in context question answering and summarisation. So, three tasks that this downstream fine-tuned large language model will have to solve.

 

What about key business outcomes? What do you expect this solution will deliver at the end of the sub-project? How do you see it, let's say, impacting your operations' competitiveness or market opportunities in score?

Marcis:

We already have interest from customers who require local solutions. But there aren't suitable solutions on the market. That's one target audience that we are trying to reach with this solution. Those are typically organisations, governmental institutions, that cannot allow data to slip outside their local infrastructure. And they particularly need this type of solution that can be deployed locally and is secure behind their own firewalls. And doesn't depend on outside services. So this particular market area we want to address and currently we don't have a solution for that. And at the end of the project, we will have it.

 

As a sole partner running this project for some months now, have you encountered any specific challenges so far, and how have you been approaching them?

Marcis:

There are two directions of challenges that we have had to deal with. One is data availability. In order to fine-tune large language models, we need a lot of data, but, there is a lot of data available, but it's available for English and not actually our languages.

We had to develop a smart method that synthesises data by using English datasets and knowledge that is available on the internet, in order to get to the datasets that solve the same problem. So, for instance, in context question answering, but in our languages, so Latvian, Estonian, Slovenian, etc.

That's one challenge that we had to deal with. So, data availability and how to actually get to the data, and the other one is accessing HPC resources. So the FFplus subproject did not guarantee access to HPC resources. So we had to apply to them. And we applied to the EuroHPC access course and those are a bit more research-tuned and not industry and commercial applications-tuned access calls, and I'll be honest. So the first application that we submitted was rejected. And it was rejected because it was more aligned to what our business needs, and not more so aligned to what would constitute cutting-edge research. Then we repackaged it, rephrased it, so that it would be a bit broader and would contain more research objectives and adapt got through.

For anyone who wants to access HPC resources that EuroHPC offers, I would suggest thinking more research-oriented than business-oriented, because that's their current evaluation strategy. So there's a higher chance that the proposals will go through. Another issue is that HPC centres and training, actually, something on HPC centres requires a bit of different thinking and approach than if you would use your own local infrastructure and, we had to adapt to that. For instance, HPC centres typically accept jobs for a specific time and we use the HPC centre Lumi in Finland. And that has a maximum role time of 48 hours.

Each job that we submit will be cancelled after 48 hours. Then we had to think of a way to submit jobs that reschedule once in a while. After every two days, a new job that training continues. And then also there's a high demand for EuroHPC resources.

There can be cases where you are not granted instances immediately. It can take hours, up to ten or more hours, to actually get to instances that are granted, and that wait time is actually idle. You are not training anything. If training of a model takes months, then you have to take into account that a week or so will probably be some downtime.

That needs to be taken into account when planning how much training we will require. Quite a lot of challenges that we had to go through, but that's a normal work process, I would say.

 

 This thought actually brings us to the end of the interview. Before we really close it, what would be your advice to the SMEs that haven't yet explored HPC and AI for their business so far?

Marcis:

Looking at our experience, I would suggest three things. First, know your data, because in order to develop any AI solution nowadays, you need to be able to have data to train a model. And it's not enough to just think that you have data. You actually need to be able to tell how much and what that data is.

And for that particular task you want to do. And that's a big, big stumbling stone for many businesses that they think they have data, but actually they don't, or they do not have processes in place that help manage, curate and , handle data.

The second is, plan ahead. If you want to use EuroHPC resources, expect that you will need a couple of months, if not more, to actually get access to those resources. That is, if you want to access EuroHPC access calls, some of those HPC centres have commercial offerings that allow immediate access.

But that is a paid service. And then, third, using HPC resources requires experience. Expect that you will need to send people for training if you don't have the expertise currently. It's not like tomorrow you will be able to use HPC resources. So training needs to be factored in when thinking about how to use HPC resources, how to develop AI solutions, etc..

 

Learn more about this innovation study here.