AI Isn't Just for Python: Integrating LLMs into Your Existing Stack
While AI discussions often assume Python, many real-world services run on Ruby, Java, and PHP. Using RubyLLM as a case study, we outline practical paths and decision-making criteria for integrating LLMs without changing your existing tech stack.
AI Isn't Just for Python: Integrating LLMs into Your Existing Stack
TL;DR You don't need Python to integrate LLMs. We look at the practical paths (integration layers, Python microservices, or direct HTTP calls) for Ruby, Java, and PHP-based teams evaluating AI adoption, along with their real-world pros and cons. No matter which framework you choose, the most critical starting point is defining exactly "what problem you want to solve."
Our Backend Runs on Ruby. Do We Need to Hire a Python Developer Just to Integrate an LLM?
Any team considering integrating AI into an existing stack has likely hit a roadblock with this question. "We know we need to bring in AI, but our service runs on Rails." This is one of the most common things our team hears in developer communities and client meetings. As AI discussions heat up, teams using Ruby, Java, or PHP can feel increasingly left behind. Tech blogs are flooded with Python examples, and LangChain tutorials and PyTorch docs all assume you are using Python.
But real-world stacks are different. A Rails-based SaaS backend that has run stably for years, an e-commerce platform running on Java Spring, or an ERP system built with PHP Laravel—this is what most companies actually run on. Do these teams really need to start with new hires just to add an LLM? That is precisely why we started digging into this question.
How Did the AI Development Ecosystem Become a Python Monopoly?
A Structural Bias Rooted in Academia and Research
It is no accident that Python dominates the AI ecosystem. Once NumPy and SciPy became the standard for scientific computing, deep learning frameworks naturally built on top of them. PyTorch started as Python-first at Facebook AI Research, TensorFlow did the same at Google Brain, and LangChain grew rapidly within the Python ecosystem. In the Stack Overflow Developer Survey 2025, Python consistently ranks near the top. However, these statistics don't represent the entire landscape of web backends.
Real-World Barriers for Teams Trying to Adopt AI Without Python
With AI frameworks solidifying around Python, non-Python teams face three practical hurdles. First is the cost of learning a new language—the time it takes for a Rails developer to learn Python and get comfortable with LangChain. Second is isolation from the existing codebase—building a separate Python-based AI service duplicates authentication, data access, and error handling. Third is operational overhead—you need a separate team to deploy, monitor, and maintain a Python microservice. How many SMBs can manage all three of these challenges at once?
A Practical Choice for Ruby LLM Integration: RubyLLM
OpenAI, Anthropic, and Gemini Under a Single Interface
RubyLLM is an open-source framework that wraps major LLM providers like OpenAI, Anthropic, and Google Gemini into a single Ruby interface. In terms of Rails AI integration, it lets you add LLM connectivity simply by installing a gem. Because it integrates completely with your existing Rails codebase, it connects naturally with ActiveRecord models, allowing you to run AI features within the same codebase without a separate Python service.
What 216 Hacker News Points Signify
RubyLLM gaining 216 points on Hacker News means more than just a number. On HN, where AI discussions lean heavily toward the Python ecosystem, this level of response for a Ruby-based LLM integration tool is a clear signal of community-level demand: non-Python teams want to use LLMs too. Of course, Hacker News points don't guarantee production readiness. We highly recommend checking GitHub star trends, issue response times, and the date of the latest commit yourself.
The 'Lowest Common Denominator' Trap: The Flip Side of Abstraction
We need to be honest here. While abstracting providers into a single interface increases convenience, it makes it harder to fully leverage provider-specific features. Features like Claude's Extended Thinking, GPT-4o's multimodal capabilities, or Gemini's context window optimizations can only be used in a limited capacity through an abstraction layer. Dynamic routing to specific providers or falling back to cheaper models for cost and performance optimization is also not yet fully resolved. Discuss these limitations within your team before committing.
We Aren't Talking About 'New AI Apps'
The Real Target: Rails-Based SaaS and E-Commerce
The true value of frameworks like RubyLLM doesn't lie in building brand-new, AI-native applications. It is about adding automatic customer inquiry classification to a Rails-based B2B SaaS that has been running for seven years, using an LLM to generate product description drafts on an e-commerce backend, or introducing an email summarization feature to an existing CRM. These are far more practical and in higher demand. This aligns with what our team hears from clients: the demand to "add an AI feature to our current service" overwhelmingly outweighs the desire to "build a brand-new AI service."
AI Integration Paths for Existing Stacks: Integration Layer vs. Python Microservice
From a practical standpoint, there are three primary paths you can take:
① Use an integration layer within your existing language (like RubyLLM)
- Pros: Seamless integration with existing codebases, minimal operational complexity.
- Cons: Limited use of provider-specific features, highly dependent on framework maturity.
② Run a separate Python microservice
- Pros: Leverages the entire Python ecosystem, allows full use of provider features.
- Cons: Duplicated operational overhead, fragmented team focus, high initial setup costs.
③ Direct HTTP calls to LLM provider APIs
- Pros: Language-agnostic, extremely straightforward architecture.
- Cons: You must build your own error handling and retry logic; handling streaming responses can be tedious.
There is no single correct path. If your team has zero Python experience and the features you want to add are relatively simple, ① is the most realistic path. On the other hand, if provider-specific features like Claude's Extended Thinking are core to your product, ② is the way to go, while ③ is great for prototyping or adding a single isolated feature. That said, whether the spread of these language-specific AI integration layers is entirely positive for the ecosystem is open to debate. If the framework wars that saturated the Python world spill over into Ruby, Java, and PHP, it could fragment community efforts in the long run. Whether this leads to "innovation" or "fragmentation" depends heavily on how active each language's community remains.
How To Choose the Right LLM Integration Path for Your Team
Decision-Making Checklist by Team Profile
If you are still wondering, "So, what should my team do?", we recommend starting with a quick internal discussion around these four questions:
- Does our team have Python development expertise? — If yes, running a Python microservice is often the more natural choice.
- Does the LLM feature we want to build rely on provider-specific features? — If multimodal processing or Extended Thinking is core, check the limits of your abstraction layer first.
- Is tight integration with our existing codebase crucial, or can it run as an isolated service? — If it needs to connect closely with your existing database and auth systems, an integration layer is better.
- Do we have the team structure to handle long-term maintenance? — If open-source framework communities shrink, the maintenance burden falls entirely on your team.
3 Criteria to Evaluate Open-Source Frameworks Before Adopting AI
Before putting any language-specific LLM integration layer into production, verify these metrics yourself:
- GitHub Star Trends: Has it been growing over the last six months, or has it stagnated/declined?
- Last Commit Date and Issue Resolution Speed: A hiatus of 2–3 months or more indicates a maintenance risk.
- Production Use Cases: Are real-world use cases shared in the community, or are there only README examples?
Without checking these three, jumping in with "let's just add the gem" can leave you stranded if support vanishes six months down the line—forcing you to fork the codebase or migrate. A quick check now can save massive costs later.
The Real Barrier to LLM Integration Isn't Your Language Stack
Our team at Grinda AI focuses on this topic not just because of tech trends. We face this daily when naturally embedding AI into existing operational systems—like ERPs, CRMs, and sales management tools—that have run in export environments for years. We know that "how to integrate AI into what we currently have" is a much more practical question than "let's build it brand new in Python." The real barrier to LLM integration isn't your language stack; it's a clear understanding of what problem you are trying to solve with AI. Without that, no framework can help you.
Written by · RINDA Export Sales Research Team (Global Buyer Discovery & Export Sales Automation Research Editors)
Drawing from global buyer discovery pipeline data of 200+ Korean exporters and internal observations from the RINDA platform, we compile immediately actionable strategies and checklists for export operations.
In a similar vein, Grinda AI helps exporters find global buyers and automate export sales by embedding AI directly into their existing sales workflows. This approach always starts with "what problem are we solving?" rather than the tech stack. RINDA, our specialized AI platform for global buyer discovery, was built on this exact philosophy. If you are curious about the most practical path to integrating AI into your team's stack, have a 30-minute chat with the Grinda AI team.
Frequently Asked Questions
Q. Does using an abstraction layer like RubyLLM make it easier to switch providers later?
A. Switching is relatively straightforward for general features like basic text generation or chat completion. However, if you are already using provider-specific features like Claude's Extended Thinking or GPT-4o vision, switching means either abandoning those features or implementing them yourself. It is strategically wiser to intentionally manage your dependency on provider-specific features from the start.
Q. What is the most important thing to watch out for when adopting AI without Python?
A. You must thoroughly verify the maturity and maintenance activity of any open-source LLM integration framework beforehand. Without checking GitHub star trends, recent commits, and real-world production use cases, you risk being stranded if support drops in six months, forcing you to fork the code or migrate. Rather than rushing in for convenience, prioritize long-term operational viability.
Q. What should be decided first before initiating LLM integration on an existing service?
A. Before asking "what features should we add?", ask "does this feature actually solve a user or business problem?" While LLM integration is getting easier technically, AI features built without a clear problem definition often end up being abandoned, leaving behind only maintenance costs. We recommend starting only when your team can clearly answer the question: "Who will suffer if this feature is gone?"