Open Source in Finance Forum London 2024

There is little doubt that (Generative) AI will have an impact on almost every aspect of our business and personal lives. However, we are at an interesting juncture: models are becoming ever more powerful, with prototypes showing ever greater promise, but there remain significant challenges when it comes to the reality of putting this technology into practice. How do we take these astonishing experiments and turn them into something safe and reliable that can ultimately be trusted in the wild? Financial services has a long history with AI and machine learning. However, its place within the industry is shifting. It is no longer the exclusive tool of Data Scientists. Rather it is a tool we are placing in the hands of our engineers, product owners, and, if we are to maximise the benefits, we should be putting it in everyone's hands. But to do-so safely, we need to educate people regarding the strengths and weaknesses, the risks and opportunities this technology presents. Within FINOS, we are forming an AI readiness group that tackles some of these challenges. This talk will take a deep dive, exploring the changes we need to make as AI rapidly becomes a standard tool in our armoury

In the rapidly evolving AI field, Large Language Models (LLMs) like GPT and BERT drive innovative applications from natural language processing to generative content, at the same time presenting unique challenges to the organisations for deploying it in a secure, trusted and compliant way.
OWASP Top 10 for LLM Applications is one of leading industry efforts about the potential security risks when deploying and managing LLMs.
This talk will review OWASP’s LLM evolving approach, using real-world scenarios to highlight critical security vulnerabilities and exploits and offer practical examples for evaluating and deploying relevant safeguards.
Key Takeaways:
- Deep insights into OWASP's top security vulnerabilities for LLMs.
- The impact of these vulnerabilities on businesses and users, underlining the need for strong security measures.
- Awareness of defensive techniques and countermeasures against LLM-specific attacks

In this talk, Jochen presents various OSS tools from NeMo LLM Framework that can be leveraged on an accelerated computing platform for building scaling, enterprise-ready (Financial) LLMs with outstanding performance, accuracy, stability and guardrailed trustworthiness. Initial step is to select one of the best community-build LLMs like Llama, Falcon, Mixtral or Nemotron. Next steps is to use tools for model customization, retrieval augmented generation, and guardrailing. Then, the model will be deployed with an inference platform that is optimized for throughput and scale. Input data can be tokenized, curated, and deduplicated at scale.

To successfully extract value from data, firms need to make sure they not only have the right financial domain knowledge, the right quantity, and quality of financial data but also a reliable data infrastructure and ML-oriented pipelines orchestrated by MLOps. With a proper MLOps strategy, domain knowledge teams can collaborate closely, properly test models, quickly respond to new data sources, and train and re-train ML models at scale while remaining compliant.

AI revolutionizes many fields, yet its success hinges on expert trust and effective interaction. This talk explores key lessons from journalism, life science, and healthcare about building explainable and human-centric AI. Discover: How journalists use AI to maintain control over news analysis amidst information overload. Principles from life science on ensuring complex AI models remain interpretable for decision-making. How healthcare experts collaborate with AI while adhering to strict regulations. Key Takeaway: Learn how these strategies can inspire new approaches to AI in Financial Services (FSI), addressing explainability, expert input, and responsible use. Leave equipped with insights to promote open-source AI development that empowers experts in FSI.

In today's financial market, data and code control are critical for success. However, dealing with sensitive data and a highly regulated market can be challenging. To address this, we decided to create an innersource VSCode extension that would give us complete control over our data and code while improving developer productivity and satisfaction. We designed this extension from day one to be innersource and deployable globally across our company. Our solution leverages our own ChatGPT based on Azure OpenAI, called SecureGPT, to provide an intuitive and easy-to-use interface for developers. As we are still experimenting with this technology, we have involved testers from across our tech teams to provide feedback and help us improve the extension. In this session, I will show you what we have built, why we chose to make it innersource from the beginning, and share some of the return of experience from our users. Join us and learn how we are using innersource and AI to improve developer productivity and satisfaction while maintaining complete control over our sensitive data and code. Leave no Dev behind, and let's build the future of finance together!

In finance, leveraging AI can offer tremendous benefits, but it also has high risks that demand security excellence. Several regulations, classification of threats and risks, mitigation guidelines, and best practices have been created to assist in that. Yet, despite these well-intentioned recommendations, they leave us with the question: How exactly do we implement these measures? During this session, Vicente Herrera will present a practical security governance architecture for GenIA, focusing on financial institutions and open source. Through a model project, he will demonstrate threats, aligning them with the OWASP Top-10 and MITRE ATLAS classifications. Then, he will show how open-source tools and techniques can effectively mitigate these risks. Live demonstrations will showcase the consequences of implementing or neglecting these mitigations, where they must be in place, and why they are important.

Join me for a dynamic exploration into the power of community-led innovation in today's tech landscape. In this talk, we dive deep into how emerging technologies like AI, DLT, and Telecommunications are shaping our world. Discover how collaborative efforts, spearheaded by Zenith (the Emerging Technology special interest group), are breaking down barriers in understanding, implementing, and commercializing cutting-edge technologies. Through captivating examples and real-world case studies, you'll gain insights into how community-driven initiatives are driving progress in the fast-paced realm of horizon tech. Don't miss this opportunity to uncover the secrets behind successful tech adoption and transformation. Let's navigate the future together!