Investing in the Stack Exchange Network and the future of Stack Overflow - Related to insights, is, fake, stack, exchange

Investing in the Stack Exchange Network and the future of Stack Overflow

2024 was an exciting year at Stack Overflow. From the launch of new products and elements that came directly from integrations with global partnerships to outlining our vision for Knowledge-as-a-service and the changing state of internet models. Hard to believe, but in many ways we have so much more to share in 2025.

As I closed out my last quarterly upgrade, I mentioned, “At Stack Overflow, we’re committed to our community, whether that be our public platform individuals, our clients, partners, and Stackers, but we also deeply believe in building a future where our larger knowledge communities can thrive in the era of AI. An essential part of that mission is preserving the trust that our community has placed in us for 16 years, and that means continuing to invest back into our knowledge ecosystem.”.

“Investing” in Community is something we discussed a number of times over the past several years as we began our efforts to evaluate how Stack Overflow will evolve in the GenAI era. For some of you, I’m sure the question was raised: “Well, how exactly are you investing in the community?” I’m glad you asked.

Within the digital walls of Stack Overflow, we frequently say “Keep Community at Our Center.” And we mean that. This manifests itself in a number of ways, and one of those is our commitment to the high-quality, accurate library of knowledge built and maintained by the Stack Exchange Communities over the past sixteen years. For Stack Overflow and the larger Stack Exchange network to continue to grow and thrive, we must invest in ways to make it easier for new users to be able to contribute high-quality content. We also need to invite contributors—new ones and long-time, experienced ones—to contribute in new forms beyond just Q&A.

Over the course of the next year, we’ll be focused on ensuring our sites remain a go-to destination for our customers, including by modernizing the existing assets we have in place and also by introducing new capabilities and capabilities that promote contribution from all types of customers. There is already a lot of experimentation at work on our team, and although we’ve tried to show previews where we can, we want to peel back the curtain and show you more.

The efforts we make and the new attributes or products we unveil mean nothing if we aren’t involving community members—the true experts on how the product is used and the challenges it has. We need more community voices at the table, so we’re actively exploring ways to bring in more community perspectives and participation. In order to be an inclusive destination for everyone who wants to participate and consume content, we need a wide range of participants in our research and testing efforts. One of the ways to invite these new perspectives will be our first public AMA.

VP of Community Philippe Beaudette, Chief Product & Technology Officer Jody Bailey, and I will share our vision for new content types and how they will include the quality controls that have made Stack Overflow the trusted resource it is. We’ll also explore other opportunities for community members to edit, rate, and build new features collaboratively.

I hope you’ll join us live on Wednesday, February 26th at 3 pm ET via our YouTube channel (here) to hear our vision. We’ll answer as many questions as we can during the time we have together.

Have you ever stumbled upon something new and went to research it just to find that there is little-to-no information about it? It’s a mixed feeling: ......

When we introduced GitHub Copilot back in 2021, we had a clear goal: to make developers’ lives easier with an AI pair programmer that helps them write......

In my previous posts, we explored how LangChain simplifies AI application development and how to deploy Gemini-powered LangChain applications on GKE. ......

Article: Launching GenAI Productivity Tools: Insights and Lessons

Key Takeaways GenAI can enhance employee productivity while safeguarding data security with data redaction and locally-hosted models.

Centralizing tools and aligning them with user behavior is critical for success.

Adopting trends like multimodal inputs and open standards can future-proof AI strategies.

Not all GenAI bets will pay off, so be deliberate with GenAI strategy and focus on business alignment.

GenAI has evolved from the initial hype to practical application and the "slope of enlightenment".

On November 30, 2022, OpenAI released ChatGPT. That release changed the way the world understood and consumed Generative AI (GenAI). It took what used to be a niche and hard-to-understand technology and made it accessible to virtually anyone. This democratization of AI led to unprecedented improvements in both innovation and productivity in many fields and business roles.

At Wealthsimple, a Canadian financial services platform on a mission to democratize financial access, there is excitement around the potential of GenAI. In this article, which is based on my talk at QCon San Francisco 2024, I will share some of the ways we're leveraging GenAI to enhance productivity and the lessons that came out of it.

Our GenAI efforts are primarily organized into three streams. The first is employee productivity. This was the original thesis of how we envisioned LLMs could add value and it continues to be an area of investment today.

As we started building up the foundations and tools for employee productivity, this gave us the confidence to optimize operations, which became our second stream of focus. Here our goal is to use LLMs and GenAI to provide a more delightful experience for our clients.

Third, but certainly not least, there's the underlying LLM platform, which powers both employee productivity and optimizing operations. We developed and open sourced our LLM gateway, which, internally, is used by over half the organization. We developed and shipped our in-house personally identifiable information (PII) redaction model. We made it simple to self-host open source LLMs within our own cloud environment as well as train and fine-tune models with hardware accelerations.

The first thing that we did in 2023 was launch our LLM gateway. When ChatGPT first became popular, the general public was not as aware of third-party data sharing as it is today. There were cases where companies were inadvertently sharing information with OpenAI, and this information was then being used to train new models that would become publicly available. As a result, many companies chose to ban employees from using ChatGPT to prevent this information from getting out.

At Wealthsimple, we believed in the potential of GenAI, so we built a gateway that would address security and privacy concerns while also providing the freedom to explore. The first version of our gateway did one thing: it maintained an audit trail. It would track what data was being sent externally, where it was being sent, and who sent it.

The gateway was available for all employees and it would proxy the information from the conversation, send it to various LLM providers such as OpenAI, and track this information. From a dropdown, consumers made selections among the different models to initiate conversations. Our production systems could also interact with these models programmatically through an API endpoint from our LLM service, which also handles retry and fallback mechanisms.

After we built the gateway, we ran into a problem with adoption: there wasn't that much incentive to use it. Our philosophy at Wealthsimple is that we want to make the right way the easy way. We used a series of sticks and carrots to improve adoption, with an emphasis on the carrots.

One of the benefits of our gateway is we made it free to use: we paid all of the API costs. Second, we wanted to create a centralized place to interact with all of the different LLM providers. At the beginning, it was just OpenAI and Cohere, but the list expanded as time went on.

We also wanted to make it a lot easier for developers. In the early days of interacting with OpenAI, their servers were not the most reliable, so we increased reliability and availability through a series of retry and fallback mechanisms, and we worked with OpenAI to increase our rate limits.

Alongside those carrots, we had some very soft sticks. The first is what we call nudge mechanisms. Whenever anyone visited ChatGPT or another LLM provider directly, they would get a gentle nudge on Slack saying: "Have you heard about our LLM gateway? You should be using that instead". We also provided guidelines on appropriate LLM use which directed people to leverage the gateway for all work-related purposes.

Although the first iteration of our LLM gateway had a great paper trail, it offered very few guardrails and mechanisms to prevent data from being shared externally. But we did have a vision centered around security, reliability, and optionality. We wanted to make the secure path the easy path, with the guardrails to prevent sharing sensitive information with third-party LLM providers.

Guided by this vision, the next thing we shipped in June of 2023 was our own PII redaction model, which could detect and redact any potentially sensitive information prior to sending to external LLM providers. For example, telephone numbers are recognized by the model as being potentially sensitive PII, so they are redacted.

While this closed a gap in security, it introduced a different gap in the user experience. Many people reported that the PII redaction model was not always accurate, which often interfered with the relevancy of the answers provided.

Secondly, for them to effectively leverage LLMs in their day-to-day work, they needed to be able to use some unredacted PII, because that was the data they worked with. Going back to our philosophy of making the right way the easy way, we started to look into self-hosting open source LLMs.

For self-hosted LLMs, we didn't have to run the PII redaction model. We could encourage people to send any information to these models, because the data would stay within our cloud environments. We spent the next month building a simple framework using [website], a quantized framework for self-hosting open-source LLMs.

Next we introduced a very simple semantic search as our first RAG API. We encouraged our developers and our end clients to build upon this API and other building blocks we provided in order to leverage LLMs grounded against our organization context.

Even though many of our customers asked for grounding, and it intuitively made sense as a useful building block within our platform, the engagement and adoption was actually very low. We realized that we probably didn't make the user experience easy enough. There was still a gap when it came to experimentation and exploration. It was hard for people to get feedback on the GenAI products they were building.

In recognizing that absence of feedback, one of the next things that we invested in was our data applications platform. We built an internal service using Python and Streamlit. We chose that stack because it's easy to use and it's something many of our data scientists were familiar with.

This platform made it easy to build new applications and iterate over them. In a lot of the cases, these proof-of-concept applications expanded into something much bigger. Within just the first two weeks of launching our data application platform, we had over seven applications running on it. Among those seven, two eventually made it into production where they're adding value and optimizing operations and creating a more delightful client experience.

As our LLM platform came together, we also started building internal tools that we thought would be very powerful for employee productivity. At the end of 2023, we built a tool we called Boosterpack, to provide employees with a personal assistant grounded against the Wealthsimple context.

Boosterpack allowed consumers to upload documents to create knowledge bases, either private or shared, with other consumers. Once the knowledge bases were created, consumers could leverage the chat functionality to ask questions about it. Alongside the question-answering functionalities, we also provided a reference link to the knowledge source. This reference link addition was really effective at providing fact check or further reading, especially when it came to documents as a part of our knowledge bases.

2023 ended with a lot of excitement. We started the year off by introducing our LLM gateway, introducing self-hosted models, providing a RAG API, and building a data applications platform. We ended the year by building what we thought would be one of our most useful internal tools ever. We were in a bit of a shock when it came to 2024.

Gartner's hype cycle maps out the evolution of expectations and changes when it comes to emerging technologies. This is very relevant for GenAI, because in 2023, most of us were entering the peak of inflated expectations.

We were so excited about what LLMs could do for us and we wanted to make big bets in this space. But as we entered 2024, it was sobering for us as a business and for the industry as a whole: we realized that not all of our bets had paid off. We then evolved our strategy to be a lot more deliberate, focusing on the business alignment with our GenAI applications. There was less appetite for bets.

The first thing we did as a part of our LLM journey in 2024 was un-shipping something we built in 2023. When we first launched our LLM gateway, we introduced the nudge mechanisms, which were the Slack reminders for anyone not using our gateway.

Long story short, it wasn't working. The same people were getting nudged over again, and they became conditioned to ignore it. Instead, what we found was that improvements to the platform itself were a much stronger driver for behavioral changes.

Following that, we started expanding the LLM providers that we supported. The catalyst for this was Gemini. Around that time, Gemini had launched their 1-million-token context window models, and we were really interested to see how this could circumvent a lot of our previous challenges with the context window limitations.

A big part of 2024 was about keeping up with the latest trends in the industry. In 2023, a lot of our time and energy were spent on making sure we had the best state-of-the-art model available on our platform. We realized that this was a losing battle, because the state-of-the-art models were changing every few weeks. Instead of focusing on the models, we took a step back and focused on higher-level trends.

One emerging trend was multimodal inputs: forget about text, now we can send a file or a picture. This trend caught on really quickly within our enterprise. We added a feature within our gateway allowing our end clients to upload either an image or a PDF, and the LLM would then drive the conversation from those inputs. Within the first few weeks of launching this tool, nearly one-third of our end clients started leveraging a multi-modal feature at least once a week.

One of the most common use cases we found was when people were running into issues with our internal tools. As humans, if you're a developer, and someone sends you a screenshot of their stack trace, that's an antipattern: you would prefer to get the text version.

While humans have very little patience for that sort of thing, LLMs embraced it. Pretty soon, we were seeing behavioral changes in the way people communicate, because the LLM's multimodal inputs made it so easy to just paste a screenshot.

Figure 2: Sending an error screenshot to an LLM.

Figure 2 demonstrates an example of an error someone encountered when working with our BI tool. This is a fairly simple error. If you asked an LLM, "I keep running into this error message while refreshing MySQL dashboard, what does this mean?" The LLM actually provides a fairly detailed explanation of how to diagnose the problem (see Figure 3).

Figure 3: The LLM Explains an Error Message.

After supporting multi-modal inputs, the next thing we added to our platform was Amazon Bedrock. Bedrock is AWS's managed service for interacting with foundational large language [website] also provides the ability to deploy and fine-tune these models at scale. There was a very big overlap between everything we had been building internally and what Bedrock had to offer.

We had considered Bedrock back in 2023, but decided instead to build these capabilities ourselves. Our motivation at that time was to build up the confidence and know-how internally, to deploy these technologies at scale.

2024 marked a shift in our build-versus-buy strategy. We're certainly more open to buying, but we have some requirements: security and privacy, first; price and time to market, second...

After adopting Bedrock, we turned our attention to the internal API that we exposed for interacting with our LLM gateway. When we first shipped this API, we didn't think too deeply about what the structure would look like, which ended up being a decision we would regret.

Because OpenAI's API specs became the gold standard, we ran into a lot of headaches with integrations. We had to rewrite a lot of code from LangChain and other libraries and frameworks because we didn't offer a compatible API structure.

We took some time in September of 2024 to ship v2 of our API, which did mirror OpenAI's API specs. We learned that as the GenAI industry matures, it's critical to think about what the right standards and integrations are.

Over the past few years, we’ve learned many lessons. and we gained a advanced understanding of how people use these tools and what they use them to do.

There is a very strong intersection between GenAI and productivity. In the surveys and the client interviews we did, almost everyone who used LLMs found that they significantly increased or improved their productivity.

Our internal usage was almost exclusively in three categories:

Programming. Almost half of the usage was some variation of debugging, code generation, or just general programming support. Content generation or augmentation: "Help me write something. Change the style of this message. Complete what I have written". Information retrieval. Much of this was focused around research or parsing documents.

We also learned a lot of lessons in behavior. One of our biggest takeaways this year was that, as our LLM tooling became more mature, we learned that our tools are the most valuable when injected in the places we do work, and that the movement of information between platforms is a huge detractor. Having to visit multiple places for GenAI is a confusing experience, and we learned that even as the number of tools grew, most people stuck with using a single tool.

We wrapped up 2023 thinking that our Boosterpack tool was going to fundamentally change the way people use GenAI. That didn't really happen. We had some good bursts in adoption and some good use cases, but it turned out we had actually created two different places for people to get their GenAI needs. That was detrimental for both adoption and productivity.

The lesson here is that we need to be a lot more deliberate about the tools we build, and we need to put investments into centralizing these tools. Regardless of what people noted they wanted, the way they use these tools will often surprise us.

Wealthsimple really loves LLMs. Across all the different tools we offer, over 2200 messages are sent daily. Close to a third of the entire organization are weekly active individuals. Slightly over half of the organization are monthly active individuals. Adoption and engagement for these tools is really great. At the same time, the feedback that we're hearing is that it is helping employees be more productive.

Furthermore, the lessons we learned and the foundations that we developed for employee productivity pave the way to providing a more delightful client experience. These internal tools establish the building blocks to build and develop GenAI at scale, and they're giving us the confidence to find opportunities to help our clients.

Going back to the Gartner hype chart, in 2023 we were climbing up that peak of inflated expectations. 2024 was a little bit sobering as we made our way down. As we're headed into 2025, I think we're on a very good trajectory to ascend that "slope of enlightenment". Even with the ups and downs over the past two years, there's still a lot of optimism, and there's still a lot of excitement for what next year could hold.

OpenAI has launched Deep Research, a new agent within ChatGPT designed to conduct in-depth, multi-step investigations across the web. Initially availa......

Slow is officially the new down. That’s a major finding of Catchpoint’s SRE investigation 2025, with 53% of study respondents agreeing with this expression, ......

This project builds on our previous NCAA game highlight processing pipeline (link below), which used a deployment script. We're now implementing a ful......

AI Is Spamming Open Source Repos With Fake Issues

AI is being used to open fake feature requests in open source repos, . So far, AI-driven issues have been reported in Curl, React, CSS and Apache Airflow.

It’s not known how widespread the issue might be, but it’s bad enough that maintainers are speaking out about it. Jarek Potiuk is a committer and PMC Member of Apache Airflow, an open source platform that allows customers to design, schedule, and monitor data pipelines. Potiuk went public about the AI-submitted requests on LinkedIn last week and spoke with TNS about his experience.

Apache Airflow maintainers noticed they had nearly double the number of issues filed one day, up to 50 from a normal run of more like 20-25. They investigated and noticed the issues seemed to be very similar but didn’t actually make sense. They began to suspect AI created these fake issues.

“Over the last days and weeks we started receiving a lot of issues that make no sense and are either copies of other issues or completely useless and make no sense,” Potiuk explained in his LinkedIn post. “This takes valuable time of maintainers who have to evaluate and close the issues.”.

Potiuk explained to us that AI submissions don’t just create more work for maintainers; they also can lead to legitimate issues being overlooked or incorrectly closed.

“We have like 30 issues a day, maybe 40, but now in 24 hours, we’ve got 30 more, so like 100% more, this means that we couldn’t make as many decisions on other things, because we had to make decisions on: Is this a good issue or bad issue?” he expressed. “Because of the very detrimental effect of it, there were at least two or three issues which were created by real people, and some of the maintainers, who are already sensitive, they closed them as spam.”.

He reviewed the issues later and noticed two to three issues that were closed but legitimate. He reopened them, but the potential to miss a real issue is there. He’s also heard from other maintainers who have experienced a similar issue with “strange” requests, although they did not have as many issues as AirFlow saw.

Potiuk pleaded with those affiliated with the AI-driven issues to explain what was happening. One submitter reached out with an apology.

The person also told Potiuk that they had been following an Outlier AI training video about using AI to submit issues to repos. The person was not aware that they were submitting to a real repo.

“Outlier. You are doing it wrong,” Potiuk wrote in a LinkedIn post that tagged Outlier. “Please stop all the people who you are tricking into creating AI-generated, completely nonsense issues in many open-source repositories.”.

Outlier is a platform that recruits subject matter experts to help train Generative AI. It’s also a Silicon Valley unicorn and subsidiary of Scale AI.

At first, Potiuk thought Outlier was trying to train AI somehow on their responses to the requests, but that turned out to be incorrect.

“Outlier. You are doing it wrong. Please stop all the people who you are tricking into creating AI generated, completely nonsense issues in many open-source repositories.”.

— Jarek Potiuk, a committer and PMC Member of Apache Airflow.

Potiuk stated Scale representatives told him they did not intend for the video viewers to file the request with the actual repos. It was supposed to be just an exercise in creating issues. They also denied they were trying to use the repos to train their AI.

“You will work on a variety of projects from generating training data in your discipline to advance these models to evaluating the performance of models,” Outlier says in its FAQ.

Scale declined an on-the-record interview, but referred The New Stack to their LinkedIn response, where George Quraishi, who handles ops at Scale AI, wrote:

“For context, we are constantly exploring new ways to train and evaluate models; coding is one area of interest. The goal of this project in particular was to teach a model how to help developers analyze issues and implement code changes — not to submit those tickets to your repo,” he wrote. “Unfortunately, a small number of our contributors misinterpreted the project requirements and took this additional step. We immediately updated the requirements to make them clearer.”.

He continued to say that Scale values the work maintainers do and that they “have absolutely no interest in purposefully submitting tickets to inconvenience maintainers.”.

This is not the first time Outlier has attracted press attention for its actions. Last summer, [website] reported that some workers had accused Outlier of being a scam after the corporation did not pay them.

It’s unlikely this problem is just caused by one AI firm. AI is being used to spam security reports as well.

The problem goes back to at least early 2024, when cURL author Daniel Stenberg wrote about it. More lately, the security developer-in-residence at the Python Software Foundation, Seth Larson, called out the issue.

“lately, I’ve noticed an uptick in extremely low-quality, spammy, and LLM-hallucinated security reports to open source projects,” Larson wrote. “The issue is in the age of LLMs; these reports appear at first glance to be potentially legitimate and thus require time to refute.”.

The issue was “distributed across thousands of open source projects and due to the security-sensitive nature of reports open source maintainers are discouraged from sharing their experiences or asking for help,” Larson wrote.

“in the recent past I’ve noticed an uptick in extremely low-quality, spammy, and LLM-hallucinated security reports to open source projects.”.

— Seth Larson, security developer-in-residence, The Python Software Foundation.

Larson pleaded with developers not to use AI or LLMs for detecting vulnerabilities.

“These systems today cannot understand code, finding security vulnerabilities requires understanding code AND understanding human-level concepts like intent, common usage, and context,” he wrote.

He also suggested a bit of thinking goes a long way.

“Some reporters will run a variety of security scanning tools and open vulnerability reports based on the results seemingly without a moment of critical thinking,” he wrote. “For example, urllib3 in the recent past received a study because a tool was detecting our usage of SSLv2 as insecure, even though our usage is to explicitly disable SSLv2.”.

Craig McLuckie, a co-founder of Kubernetes and now founder and CEO of Stacklok, told TNS that his team had discovered someone trying to ambush repos by creating packages with similar names to well-known packages.

They discovered someone was trying to scam the tea protocol, which is a decentralized framework for managing recognition and compensation for open source software developers.

“They were publishing thousands and thousands and thousands of packages, with the sole intent of making those packages look like they were an crucial part of the open source ecosystem,” McLuckie mentioned. “Just the volume of these ambush packages, it’s just going through the roof, and it seems to me that, like, for someone to produce the volume and the sort of slight variations that we’re seeing, there’s probably a generative AI agent behind the scenes.”.

He spoke with the tea protocol developers, who agreed it was “definitely bad behavior,” then worked with npm to take the packages down.

McLuckie suspects a state actor was behind the submissions.

“Increasingly, there’s generative AI being used to create light variations on something and just doing that at scale, and I think it’s only going to get worse,” he expressed.

A GitHub engineer posted to Potiuk’s LinkedIn thread that they were looking into the issue, so TNS asked GitHub about it’s response to the problem of AI submissions to repos.

“GitHub hosts over 150M developers building across over 420M repositories, and is committed to providing a safe and secure platform for developers,” a spokesperson told TNS. “We have teams dedicated to detecting, analyzing, and removing content and accounts that violate our Acceptable Use Policies.”.

GitHub added that they employ manual reviews and at-scale detections that use machine learning and constantly evolve and adapt to adversarial tactics.

“We also encourage consumers and community members to study abuse and spam,” the spokesperson stated.

Potiuk also suggested maintainers continue to investigation AI submissions to GitHub. He also advised open source groups to work with “good” AI companies to identify fake issues. His team is working with an AI business called Dosu, which he has found helpful for sorting through issues. It’s a very different experience because the AI business is working closely with the team, he added.

“They automatically assign labels to the issue based on the content that people create, and that allows us to classify the issues without spending a lot of time,” he told TNS. “They talked to us. We had calls with them, and they explained it to us, and they gave it to us for free to do open source projects.”.

TNS Senior Editor Joab Jackson contributed to this article.

When you log in to your favorite streaming service, first impressions matter. The featured content should instantly lure you into binge-watching mode.......

AI-Driven Automation: AI will automate API lifecycle management, enhancing performance and security. API-First Development: Designing A......

Cummins: I'm Holly Cummins. I work for Red Hat. I'm one of the engineers who's helping to build Quarkus. Just as a level set before I star......