Thanks to artificial intelligence and deep learning technology, it’s now possible for any of us to insert ourselves into our favourite movie scene or create audio or video content in which we speak ten different languages. This tech brings many benefits in areas like education, entertainment and communication, but it also comes with a sinister side. The same tools can be used to create ‘deepfakes’, images of fake events or voice clones of people that can leave financial institutions vulnerable. Through the use of deepfakes, imposters could potentially outwit a bank’s security systems, posing as legitimate clients using doctored video or audio. This could lead to different forms of identity fraud, including account takeovers, application fraud and credit fraud.
What can banks do to counteract this threat? SwissHacks, the hackathon organised by the Swiss Financial Innovation Desk (FIND) and startup incubator Tenity, gave participants 48 hours to ‘hack the future of finance’ by finding a solution to this and other challenges facing the financial sector – as well as show off their own flair for quick-thinking and innovation.
Sniffing out the real from the fake
Julius Baer was one of five ‘Innovation Rockstars’ from the Swiss financial centre – alongside PostFinance, SIX, Microsoft/Unique and Ripple – who partnered the event and set challenges for the hackers. Pitching Julius Baer’s challenge, Matthias Plattner, Head Channels & Digital Services, explained that a team of Julius Baer engineers had compiled audio files of fictive client conversations. The hackers’ task was to identify the deepfake calls and verify the client’s identity based on factual content, by finding an innovative way to embed a ‘Fake Sniffer’ seamlessly and conveniently into the bank’s tech landscape.
For Matthias, there was personal inspiration behind the playful name of the targeted solution. “My wife trains dogs for search and rescue missions, so that’s where Fake Sniffer comes from,” he said. “A rescue dog has very fine nerves in its nose that can detect fakes in a millisecond using only very few clues. We want the hackers to visualise their solution the same way.” Matthias also grabbed the hackers’ attention by explaining that this challenge not only impacted the financial services sector but addressed a market of six billion people. “Finding a solution to this challenge is not only about helping banks like ours protect clients and safeguard employees, but ultimately about creating a solution that protects all of us as a society.”
Exciting tech opportunities in private banking
Hackathons have evolved in recent years from niche gatherings for the tech-savvy to become hotbeds of innovation, providing a platform for tech enthusiasts to network and exchange ideas. Open to anyone, they typically draw in students, entrepreneurs and young professionals from the fields of software design or product and business development – attracted by the opportunity to develop skills such as problem-solving, critical thinking, teamwork and time management.
For the participating companies, they not only provide a platform for scouting the next big idea but also the next generation of tech talent. “Taking part in events like SwissHacks shows that our bank is a cool place to work for if you love technology, innovation and great ideas,” says Andreas Fahrni, Julius Baer’s Head of IT. “Private banking may not always have had the reputation as being a home for tech innovators in the past, but now we’re exploring new technologies like AI, blockchain and digital assets in our tech and business divisions and doing many interesting things. To do this, we need the kind of talent that’s been on display during the past 48 hours at SwissHacks.”
Andreas says that the concept behind SwissHacks clearly demonstrates the benefits of collaboration. “By bringing people together in a focused environment to explore issues and devise quick solutions, you often achieve more than if you sat somebody in a room for two months to theorise on the issue from every angle.” What do tech leaders like him look for in the hackers’ solutions? “Above all, it’s about creativity,” he says. “We don’t want a copy-paste of something that’s already out there. We’re also interested in how people break a problem down into solvable parts to create a successful outcome in a short timeframe.”
Winning detectives solve the case
Given the high number of teams who chose to participate in Julius Baer’s deepfake conundrum, the challenge was to select a winner from the high-quality solutions presented. The jury eventually settled upon FraudFence, an audio detection and verification tool, developed by the team ‘Les Détectives’, which banks like Julius Baer can use to provide a shield against deepfakes and imposters.
Olivier Jérémy from the winning team demonstrated on stage how the FraudFence prototype turns deep learning techniques against fraudsters. “Our tool transcribes and translates the audio in real time. Then we run large language model (LLM) inference on the data provided,” he explained. “When a client calls, smart indicators appear on the tool’s dashboard that enable the relationship manager to verify the client’s identity in real-time. What’s more, the solution is hosted on the bank’s premises, so it’s compliant with global data protection regulations.”
Innovation - a mix of working hard and playing hard
After the high-paced, fun and in some cases sleep-deprived marathon of creativity over the 48 hours, what was the participants’ verdict on SwissHacks 2024?
Martin Selinga, a master’s student in Management, Technology and Economics at Zurich’s ETH, has taken part in hackathons previously but says SwissHacks stood out because it combined the “nerdy, technical stuff” with real-world business problems. “It’s not just about tinkering with Linux and coding, but more like a start-up competition.” He says the participants’ different backgrounds and cross-disciplinary experience added to the learning effect. “My teammates ranged from students to high-calibre professionals. It was a great chance for us all to gain new skills. I met many amazing people and made lots of new connections. It was an unforgettable experience.”
Matthias Plattner has invited all the participants of the Julius Baer challenge to the bank to develop their solutions further – by also leveraging the great ideas from other teams. He says events like these inject new vitality into his own team’s ideation and problem-solving efforts. “Innovation can and should happen from within, of course, but we bankers tend to think in our boxes. SwissHacks has given us a fresh perspective and energised us to transform our own sometimes hazy ideas into functional solutions.”
Although he admits he spent his weekends differently when he was a student, Matthias says the intensity of the experience mirrored his own team’s experience by providing a reminder that, notwithstanding all of the ideation and creativity on show, “successful innovation is ultimately 1% innovation and 99% transpiration!”