Artificial Intelligence (AI) Research in Ireland

How many times have you sat in a traffic jam in a car or bus, or waited seemingly for ages to cross the road as a pedestrian? The sequencing of traffic lights plays an important part of keeping people moving in a city, and researchers on the ENABLE Research Programme at Connect, the SFI Research Centre for Future Networks, are developing ways for traffic lights to ‘learn’ the most efficient sequences. 

Dr Ivana Dusparic, Ussher Assistant Professor at the School of Computer Science and Statistics at Trinity College Dublin explains, “At the moment, traffic engineers and software developers figure out the best way to program the traffic lights at junctions. We want to develop artificial intelligence to assist with that.”

To help the AI system to learn, the researchers start small. They build a computer simulation that has at its focus a single junction, and they allow the software to figure out how to work well with the junctions that connect with it. “We tell the system what we want at a very broad level – traffic flowing, no crashes – and we let it try out lots of options in the simulation. When it gets the desired result, this reinforces the machine learning.”

Then the puzzle gets a little harder: more junctions are added and the AI system figures out how to keep things flowing efficiently. So far, the project has simulated tens of city junctions based on real locations in Dublin city centre and along Cork’s Western Road. “We anticipate that in the future the system could assist traffic engineers when they are managing the junctions."

She is also using a similar machine-learning technique to keep power flowing efficiently to people’s homes and businesses.“We are applying AI to scheduling use of large domestic appliances, to co ordinate with each other and use energy when it is available at the cheapest and most sustainable rates. This is going to become increasingly relevant as we have more renewable energy on the grid, and rates may change by the hour. The idea is that as a consumer you can say you want to have your laundry dried by dinnertime and your electrical vehicle charged by morning, and the AI will work out the optimal time to automatically switch on the charger or appliance to make that happen.”

When a person gets an injury – perhaps an athlete damages a knee or a person is recovering from surgery – physiotherapy is often an integral part of their recovery. It’s important to carry out the exercises correctly, but it’s usually not realistic for a physiotherapist to be by your side checking your form as you recover, particularly at home.

That’s why Dr Georgiana Ifrim and colleagues at the Insight Data Analytics SFI Research Centre are using machine learning to develop a smart mobile app to assess how a person carries out such exercises and to give appropriate feedback.

Developing the app involves AI using ‘time-series’ data, explains Dr Ifrim, who is an SFI Funded Investigator at Insight Data Analytics SFI Research Centre and an Assistant Professor at University College Dublin School of Computer Science. “We are all pretty familiar with smart watches and other devices that keep track of your steps and heartbeats over time, and when we analyse this sequence of data, we can see patterns that tell us about a person’s activity and sleep and health. In a similar way, time series data, that is data collected over time from a sensor, can tell you about how a person moves through space when they are exercising, and we are interested in the patterns that tell us about injury and recovery.”

Dr Ifrim’s group is working with physiotherapists and engineers in UCD led by Professor Brian Caulfield (in particular with Dr Martin O'Reilly and Dr Darragh Whelan) to put sensors on athletes and on people with injuries and then use sensor and video data to track their movements as they do specific exercises. With the data, Dr Ifrim is using machine learning to build an app and sensor system that a person can use at home to take a video of themselves as they carry out exercises and the computer can assess their form and progress.“We have been getting lots of volunteers into the lab to do the exercises with the sensors and generate data, and the artificial intelligence models we develop are learning about the correct and the inappropriate ways to carry out those specific exercises.”

One of the big challenges the team wants to overcome through the research though, is that this kind of predictive models typically require lots of computational power. This could cause the mobile phone to freeze when the AI app is running. Their approach is to build a simpler, optimised and ‘explainable’ AI system that is lighter on its feet. “When we overcome such challenges, the goal is to have a mobile app that people can use easily in their homes and it will assess them and give them feedback as to whether they are doing the exercise properly. And if they are not doing the exercise properly, then it would tell them what they should change about it to keep them on track for recovery.”

This is vitally important as the team look to move the research beyond the lab setting and put it into the hands of physiotherapists and strength coaches.“To that end, Dr O’Reilly and Dr Whelan have formed the company Output Sports, who use this sensor data to test and track performance in a sports setting. They expect to launch their first product before the end of 2019.”

Telegraph poles - you probably pass them all the time, yet barely notice them unless they have fallen. For eir, though, keeping track of those all-important poles and the connections they enable is crucial. That’s why they asked researchers at SFI Research Centre ADAPT, Trinity College Dublin, to develop artificial intelligence that can help them improve their database of poles.

“Eir has a digitised inventory of poles, but they wanted to check the information was correct, they knew there could be errors,” explains Professor Rozenn Dahyot, an Associate Professor in Statistics at Trinity College Dublin.

The team at SFI Research Centre ADAPT investigated approaches to improve the quality of data about telegraph poles without the need for humans to be physically present at the pole location, but they found that some perspectives would not work well.

“We couldn’t use data from satellite images or Lidar 3D data, because when you look from above the part of the pole you see is very small and the resolution was not good enough to be able to tell much,” explains Dr Eamonn Kenny, ADAPT Research Fellow responsible for data management in the project.

Instead, they found that Street View imagery offered a more useful dataset, because it contained images of roads captured at car level. “This gave us a 360-degree view from each point, and we had GPS information about where the car was as well as which views were north, south, east and west. It contained a lot of useful information.”

One of the challenges in the project though was to ensure that the same pole wasn’t counted multiple times in the street views. The team developed artificial intelligence algorithms or computer code to avoid this, as Professor Dahyot explains, “It was important to ensure that each pole was uniquely identified across several images, we used deep-learning and other techniques from statistics and machine learning to optimise the accuracy of the results.”

To find out how well the new computer programs worked, they ran them on areas where the answers were already known, explains Dr Vladimir Krylov, ADAPT Research Fellow responsible for the AI image processing pipeline on the project. “We were given access to the records that eir had for areas where they knew the information was accurate. So we ran the algorithms in those areas to calibrate the system.”

The year-long AIMapIT project (Automatic Detection and Geotagging of Stationary Objects from Street Level Imagery) was hailed as a great success, with former eir CTO Helene Graham suggesting that the economic value of the initial use had been estimated at €3m over a three-year period. The research was also shortlisted for an IBEC Digital Technology Award Ireland in 2017  for Outstanding Academic Achievement and For the AI Awards Ireland 2018 under the title of Best Contribution To AI – Academic Research Body.

The ADAPT team has been working with Enterprise Ireland to explore the feasibility of commercialising the approach for use in other situations, explains Professor Dahyot, who sees the potential for many other applications. “There are lots of uses for the precise geolocation and tagging of objects, including 3D-mapping technologies, road signage, autonomous vehicles and even drone deliveries: you don’t want the drone delivering your package to the neighbour!”

The road to fully self-driving cars still has many miles to go, but human drivers are already benefiting from helpful technologies that allow cars to ‘sense’ the surrounding environment. Perhaps your car has a rear-view camera or other sensors that help to guide you when you park, or wipers that glide automatically at the first spatters of rain.

Information from such automatic ‘assistants’ on cars could help to boost driver safety, including anticipating hazards on or near the road, and researchers at Lero, the SFI Research Centre for Irish Software are seeking to find out how.

Dr Martin Glavin, a researcher with Lero and the Co-Director of the Connaught Automotive Research (CAR) Group at NUI Galway explains, “We look at how to interpret data from sensors on cars so that the information can be used by artificial intelligence systems. The reason we need to do this research is that the world is complex, situations and environments change and even the same route may look quite different from one day to the next.”

Dr Glavin describes how the light entering a camera on a car may be different depending on whether it is sunny, cloudy or raining outside, and how pedestrians may be wearing T-shirts or carrying umbrellas (or both!) depending on the weather. The combination of these and other factors makes detection of pedestrians a significant challenge for a vehicle moving at speed. “We use LIDAR scanning, Radar, ultrasonic technology and different types of cameras to assess what works under lots of different and challenging environments. Our research is focused on how to deliver the right blend of information from the sensors on a car to the AI systems that can then process it.”

The CAR group in NUI Galway works closely on the research with Valeo, a major supplier of sensing technology to the car industry. Dr Glavin and CAR Group Co-Director Dr Edward Jones have had a long association with Valeo’s R&D operations in Tuam, and together they are developing a ‘lab on wheels’ in the form of a Mercedes E-class kitted out with a range of sensors and scanners. “We are developing a test track on the university campus that will also be fitted with sensors, and this gives us the opportunity to test out scenarios that could be hazardous. For example, if the car is behind a bus, then someone gets off the bus and walks around it, the car can’t see through the bus, but a sensor at a traffic light can ‘see’ the pedestrian, send that information to the car and alert the driver. In that way the infrastructure and the car work together to make the roads safer.”

Who among us hasn’t enjoyed a video online of a pet being cute or a comedian saying something funny? But what if there is nothing cute or funny about a video that is live-streamed? What if an upsetting, harmful or hate-inciting event is live-streamed and shared on social media across the world? At the moment, artificial intelligence (AI) cannot reliably detect such toxic live-streams being posted or shared online. But research into video qualities and classification at Insight Data Analytics SFI Research Centre, Dublin City University, could help to solve that problem in the future.

“Right now, humans are employed to moderate content on social media platforms, but it is a dispiriting job and it is not scalable,” says Professor Alan Smeaton, a Founding Director of Insight Data Analytics SFI Research Centre and Professor of Computing at Dublin City University. “AI will offer a way to detect this kind of content, but there are still a lot of challenges to overcome, and this is where the research is vital.”

Some of those challenges include the diversity of the content and backgrounds in such videos, as well as the need for new approaches so that computers can ‘caption’ or label what is in the video itself. The issue with AI seeking out live-streams online is that the videos tend to be diverse and in different locations, therefore relatively unpredictable, notes Professor Smeaton, and even if a video has been identified as toxic, making even slight changes to it could hide it from current computer algorithms searching for it as it is being shared online.

In addition, the labeling of videos needs a new and more nuanced approach. “At the moment, we are very good at getting computers to recognise objects in videos - this is a person, this is a teapot, this is a flower. But those objects tend to be in a benign context and they are not trying to fool the AI. So our detection approaches need to work under adversarial conditions, where we’re fighting to discover things that are deliberately being masked or hidden.”

These and other hurdles that mean AI needs more research to become reliably capable of detecting toxic live-streams online, and DCU is on the case. “At Insight Data Analytics SFI Research Centre in DCU we have been working on projects to understand what makes videos interesting and memorable to viewers. We have been able to develop algorithms or computer programs to detect emotion in videos, and we have researched how  humans consume a video, what are our eyes drawn to on screen and what kind of composition affects its aesthetics. All of those things provide further ways to label a video more insightfully with ‘metadata’ that can be used by AI, and this information could be important for detecting toxicity in the content.”

The area of content moderation online is a good example of where humans still carry the burden of the job for now, and the current AI can’t take that burden at scale, notes Professor Smeaton. “I think sometimes there is a perception that we can just throw AI at almost any problem and it will fix it, but as we can clearly see here, that’s not the case. However we would hope that in years to come, having this better understanding of video will support future algorithms that can help to tackle toxic content.”

Having access to on-the-ground information can help you plan more effectively and find richer answers to questions. Ordnance Survey Ireland (OSi) has a vast amount of geospatial information about Ireland, and now, through a collaborative project with the SFI Research Centre ADAPT, the organisation is ensuring that people, software applications and AI can use this data more effectively.

“OSi has over 50 million unique geographic identifiers (UGIs) in its database, including boundaries and buildings points,” says Professor Declan O’Sullivan, a co-Principal Investigator at SFI Research Centre ADAPT and an Associate Professor at Trinity College Dublin. “OSi and ADAPT are working together to make this data accessible through a concept called 'Linked Data'. The idea is that you make the data available through the web, so people don’t have to invest in new technology to access it. Each piece of a data has a unique identifier on the web, so if a browser uses the address, the data is provided as a web page; and if a piece of software uses the identifier the data is provided in a way that it can be used directly.  This all makes the data much more accessible and information-rich both for humans and for AI.”

The collaboration between SFI Research Centre ADAPT and OSi began in 2016, first linking the data about county, local and electoral boundaries to make the information more openly accessible. The work has since continued with data about the location, form and function of buildings.

The high quality of the Linked Data provided by OSi will enable the development of a wide range of innovative applications. “OSi has the national mandate to ensure that the published, geospatial data is authoritative. This is critical for many applications especially those that use AI, such as services related to electricity and telecoms infrastructure and services that will enable innovative integration of data about a community or region with data about the physical environment. By carrying out this research, we have made OSi's high-quality geospatial data more available, so there is greater opportunity to use and innovate with it.”

The value of the work on linking the open data about geo-located objects in Ireland has quickly become apparent both at home and internationally, notes Lorraine McNerney, Chief Information Officer with OSi. “The project with ADAPT yielded the first five-star Linked Data the Irish Government has produced, and contributed to Ireland achieving 1st place in the European Open Data Maturity Index for 2017 and 2018. The research has also resulted in academic impact with published papers; it has seen many students train using the OSi data; and it has generated new collaborations.”

Open Linked Data also stands to streamline efficiencies in public and Government services. McNerney adds: “One of the key elements in the Irish Government’s Public Service Data Strategy is to remove or avoid duplicated data, and to manage data with good governance. Combining geospatial data with other data such as population statistics will continue to help Government make more evidence-based, well-informed decisions.”