Why Businesses are So Focused on Open Source AI

Transparency, reproducibility, access to diverse data, and ease of integration are the top reasons for favoring open source AI, according to a recent survey.

Photo by Igor Omilaev on Unsplash

By Irving Wladawsky-Berger

Generative AI has the potential to “profoundly alter diverse sectors by synthesizing vast amounts of data and generating new outputs.” That was the conclusion of the “2023 Open Source Generative AI Survey Report,” published by the Linux Foundation (LF) in December. “From creating intricate artworks and composing music to designing novel pharmaceutical compounds and simulating realistic human language, the potential applications of GenAI are vast and transformative.”

Specifically, the Foundation emphasized the need for open source GenAI because it is “rooted in principles of transparency, collaboration, and shared innovation.” As such, it “holds transformative potential for the advancement of GenAI technologies.” The report says that by democratizing access to AI algorithms and datasets, open source initiatives allow a broad and diverse pool of developers to contribute to, refine, and critique GenAI systems.

This collective intelligence accelerates the pace of innovation and uncovers and rectifies biases or vulnerabilities that might otherwise go unnoticed in closed development environments.”

Open Source Considerations

The definition of open-source software has been around for about 25 years. But open-AI systems require distinct definitions, protocols, and development processes because AI systems don’t behave like traditional software. The Open Source Initiative (OSI) is currently developing an Open Source AI Definition that’s being reviewed with the AI community.

What does all of that mean for businesses and developers? To help understand the potential impact of open source GenAI on the market, LF Research and LF AI & Data launched a worldwide survey to explore the current state of GenAI technologies in companies, including models, databases, applications and frameworks.

“The level of openness can vary greatly between the different GenAI models currently available, but most of them would likely not earn the open source title, since availability and access to the underlying code, data, model, and documentation are rare,” said the LF Survey Report.

The survey received 284 responses, 92% from the U.S. and Canada. Respondents’ attitudes can be summed up in these three points:

1. Businesses are concerned by the openness of the GenAI technologies they are using. “Around two-thirds of respondents are either extremely or moderately concerned about this aspect, reflecting the importance of transparency and control in technology deployments.”
2. Survey respondents generally lean in the direction of open source. “This finding highlights a recognition of the benefits associated with open source technologies, including transparency, reproducibility, access to diverse data and models, and ease of integration.

Security, an important concern for any technology deployment, does not appear to be a deterrent for open source GenAI adoption.”

3. A neutral governance approach is key to GenAI development. “Neutral governance is not only crucial for fostering responsible growth of GenAI but also for ensuring that its benefits are widespread and aligned with societal values. This approach is vital in maintaining the integrity and sustainability of GenAI advancements, ensuring that they serve both communities and stakeholders.”

A Deeper Look at the Survey

Survey participants were generally very familiar with AI and open source, as follows:

· 88% worked for companies extremely or very reliant on open source software (OSS))

· 37% worked for IT companies,

· 86% were extremely or very familiar with their organization’s adoption of GenAI.

· 31% of respondents were AI or ML engineers, 20% were non-IT senior/executive managers, 17% were in IT management, 13% data scientists, 6% product managers, 6% marketing/communications, and 4% developer/software engineer.

The questions drew detailed responses abou the importance of GenAI, how far along their efforts are, how much they’re spending and exactly how it will be deployed. Among the highlights are the following data points:

How important is GenAI to the future of the company you work for? Extremely important — 21%; moderately important — 42%; slightly important — 25%; neither important or unimportant — 3%; unimportant — 9%.

To what extent is your company involved with GenAI? Extremely involved — 31%; very involved — 49%; involved — 14%; slightly involved — 5%; not involved at all — 1%.

What percentage of the overall IT budget will your company invest in GenAI in the next 12 months? Almost entirely focused on GenAI — 9%; a large percentage encompassing several projects — 51%; a moderate portion of the budget — 29%; a small percentage for pilot projects — 8%; and no investments at all — 0%.

The top three areas where organizations expects to develop and use GenAI: Quality assurance — 35%; software testing — 34%; documentation of software and applications — 34%.

These are followed by cybersecurity analysis and mitigation — 31%; software development — 29%; marketing, sales collateral and articles — 23%; customer service, support and recommendations — 20%; access to company knowledge and data — 20%; language understanding and translation -19%; customer satisfaction analysis — 14%; personal assistants — 14%; education and training — 13%; research — 11%; and finance — 11%.

Where does your company use or plan to use GenAI technologies? Enhance internal processes — 16%; embed into existing products and services — 29%; and create new products and solutions — 55%.

How does your company employ or plan to employ GenAI technologies? Out of the box with little or no customization — 13%; extensive customization to fit company’s needs — 57%; and develop in-house GenAI technologies — 30%.

If your company does not deploy GenAI systems in the next 12 months, what are the primary reasons? Security — 49%; cost — 33%; technology maturity — 31%; no in-house AI expertise — 17%; no compelling business case — 14%; and does not apply to us — 12%.

Does your organization prefer proprietary or open source AI models? Prefer open source — 41%; 9% prefer proprietary — 9%; and both types are fine — 50%.

How much would your organization’s data control and transparency change if the GenAI technologies were open source? Significantly increase — 14%; moderately or slightly increase — 55%; stay the same — 8%; moderately or slightly decrease — 16%; significantly decrease — 2%.

This blog first appeared March 3, here.