Microsoft open-sources a crucial algorithm behind its Bing Search services

Microsoft today announced that it has open-sourced a key piece of what makes its Bing search services able to quickly return search results to its users. By making this technology open, the company hopes that developers will be able to build similar experiences for their users in other domains where users search through vast data troves, including in retail, though in this age of abundant data, chances are developers will find plenty of other enterprise and consumer use cases, too.

The piece of software the company open-sourced today is a library Microsoft developed to make better use of all the data it collected and AI models it built for Bing .

“Only a few years ago, web search was simple. Users typed a few words and waded through pages of results,” the company notes in today’s announcement. “Today, those same users may instead snap a picture on a phone and drop it into a search box or use an intelligent assistant to ask a question without physically touching a device at all. They may also type a question and expect an actual reply, not a list of pages with likely answers.”

With the Space Partition Tree and Graph (SPTAG) algorithm that is at the core of the open-sourced Python library, Microsoft is able to search through billions of pieces of information in milliseconds.

Vector search itself isn’t a new idea, of course. What Microsoft has done, though, is apply this concept to working with deep learning models. First, the team takes a pre-trained model and encodes that data into vectors, where every vector represents a word or pixel. Using the new SPTAG library, it then generates a vector index. As queries come in, the deep learning model translates that text or image into a vector and the library finds the most related vectors in that index.

“With Bing search, the vectorizing effort has extended to over 150 billion pieces of data indexed by the search engine to bring improvement over traditional keyword matching,” Microsoft says. “These include single words, characters, web page snippets, full queries and other media. Once a user searches, Bing can scan the indexed vectors and deliver the best match.”

The library is now available under the MIT license and provides all of the tools to build and search these distributed vector indexes. You can find more details about how to get started with using this library — as well as application samples — here.

Briq, the next building block in tech’s reconstruction of the construction business, raises $3 million

Bassem Hamdy has been in the construction business for a long time.

He spent the last few years at the construction software business Procore, now a $3 billion company developing technology for the construction industry, and now Hamdy is ready to unveil his next act as chief executive and co-founder of Briq, a new software service for the industry.

Hamdy started Briq with his own cash, amassed through secondary sales as Procore climbed the ranks of startups to reach its status as a construction industry unicorn. And the company has just raised $3 million in financing to fund its expansion.

“With enough secondaries you can afford to make your own decisions,” Hamdy says. 

His experience in construction dates back to his earliest days. Hailing from a family of construction engineers, Hamdy describes himself as a black sheep who went into the financial services industry — but construction kept pulling him back.

Beginning in the late nineties with CMIC, which was construction enterprise resource planning, and continuing through to Procore, Hamdy has had success after success in the business, but Briq is the culmination of all of that experience, he says. 

“As much as data entry helps people it’s data intelligence software that changes things,” says Hamdy. 

Briq chief executive Bassem Hamdy

The Santa Barbara, Calif.-based company is part of a growing number of Southern California technology startups building businesses to service large swaths of specific industries — specifically real estate and construction.

Already, Procore is a $3 billion behemoth, and ServiceTitan has become a billion-dollar company as well, with its software and services for air conditioning and appliance repairmen.

Now Hamdy’s Briq, with backing from Eniac Ventures and MetaProp NYC, is hoping to join their ranks.

“Bassem built and helped run the most successful construction software businesses in the world. It is rare and humbling to have an opportunity to help build a company from the ground up with an industry legend,” says Tim Young, founding general partner at Eniac Ventures . “The technology Bassem and his team are building will do something the industry has never seen before: break down data silos to leverage information in real time. Bassem has built and run the most successful construction software businesses in the world, and his knowledge of the construction space and the data space is second to none.”

The company, formerly called Brickschain, uses a combination of a blockchain-based immutable ledger and machine learning tools to provide strategic insights into buildings and project developments.

Briq’s software can predict things like the success of individual projects, where demand for new projects is likely to occur and how to connect data around construction processes.

Briq has two main offerings, according to Hamdy. ProjectIQ, which monitors and manages individual projects and workflows — providing data around different vendors involved in a construction project; and MarketIQ, which provides market intelligence around where potential projects are likely to occur and which projects will be met with the most demand and success.

Joining Hamdy in the creation of Briq is Ron Goldschmidt, an experienced developer of quantitative-based trading strategies for several businesses. Hamdy, a former Wall Streeter himself, has long realized the power of data in the construction business. And with the new tools at his disposal — including the blockchain-based ledger system that forms the backbone of Briq’s project management software, Hamdy thinks he has developed the next big evolution in technology for the industry.

Briq already counts Webcore, a major contractor and developer, as one of its clients, along with Kobayashi, Probuild, Hunter Roberts OEG and Gartner Builders. In all, the company has contracts with nearly 12 developers and contractors.

All of the insights that Briq can provide through its immutable ledger can add up to big savings for developers. Hamdy estimates that there’s roughly $1 trillion in waste in the construction industry.

Briq relies on IBM’s Hyperledger for its blockchain backbone and through that, the company has a window into all of the decisions made on a project. That ledger forms the scaffolding on which Briq can build out its projections and models of how much a building will cost, and how could conceivably be made on a project.

“Construction and infrastructure are integral to society, but the decision-making process behind how, when, where, and why we build is no longer working,” said Hamdy, in a statement. “We aren’t just solving a construction problem, we are solving a societal problem. If we are to meet the infrastructure needs of both the developed and developing world, we must improve our decision-making and analysis around the data we have. We are thrilled to have the support of Eniac Ventures as we enter the next phase of our journey.”

Briq, the next building block in tech’s reconstruction of the construction business, raises $3 million

Bassem Hamdy has been in the construction business for a long time.

He spent the last few years at the construction software business Procore, now a $3 billion company developing technology for the construction industry, and now Hamdy is ready to unveil his next act as chief executive and co-founder of Briq, a new software service for the industry.

Hamdy started Briq with his own cash, amassed through secondary sales as Procore climbed the ranks of startups to reach its status as a construction industry unicorn. And the company has just raised $3 million in financing to fund its expansion.

“With enough secondaries you can afford to make your own decisions,” Hamdy says. 

His experience in construction dates back to his earliest days. Hailing from a family of construction engineers, Hamdy describes himself as a black sheep who went into the financial services industry — but construction kept pulling him back.

Beginning in the late nineties with CMIC, which was construction enterprise resource planning, and continuing through to Procore, Hamdy has had success after success in the business, but Briq is the culmination of all of that experience, he says. 

“As much as data entry helps people it’s data intelligence software that changes things,” says Hamdy. 

Briq chief executive Bassem Hamdy

The Santa Barbara, Calif.-based company is part of a growing number of Southern California technology startups building businesses to service large swaths of specific industries — specifically real estate and construction.

Already, Procore is a $3 billion behemoth, and ServiceTitan has become a billion-dollar company as well, with its software and services for air conditioning and appliance repairmen.

Now Hamdy’s Briq, with backing from Eniac Ventures and MetaProp NYC, is hoping to join their ranks.

“Bassem built and helped run the most successful construction software businesses in the world. It is rare and humbling to have an opportunity to help build a company from the ground up with an industry legend,” says Tim Young, founding general partner at Eniac Ventures . “The technology Bassem and his team are building will do something the industry has never seen before: break down data silos to leverage information in real time. Bassem has built and run the most successful construction software businesses in the world, and his knowledge of the construction space and the data space is second to none.”

The company, formerly called Brickschain, uses a combination of a blockchain-based immutable ledger and machine learning tools to provide strategic insights into buildings and project developments.

Briq’s software can predict things like the success of individual projects, where demand for new projects is likely to occur and how to connect data around construction processes.

Briq has two main offerings, according to Hamdy. ProjectIQ, which monitors and manages individual projects and workflows — providing data around different vendors involved in a construction project; and MarketIQ, which provides market intelligence around where potential projects are likely to occur and which projects will be met with the most demand and success.

Joining Hamdy in the creation of Briq is Ron Goldschmidt, an experienced developer of quantitative-based trading strategies for several businesses. Hamdy, a former Wall Streeter himself, has long realized the power of data in the construction business. And with the new tools at his disposal — including the blockchain-based ledger system that forms the backbone of Briq’s project management software, Hamdy thinks he has developed the next big evolution in technology for the industry.

Briq already counts Webcore, a major contractor and developer, as one of its clients, along with Kobayashi, Probuild, Hunter Roberts OEG and Gartner Builders. In all, the company has contracts with nearly 12 developers and contractors.

All of the insights that Briq can provide through its immutable ledger can add up to big savings for developers. Hamdy estimates that there’s roughly $1 trillion in waste in the construction industry.

Briq relies on IBM’s Hyperledger for its blockchain backbone and through that, the company has a window into all of the decisions made on a project. That ledger forms the scaffolding on which Briq can build out its projections and models of how much a building will cost, and how could conceivably be made on a project.

“Construction and infrastructure are integral to society, but the decision-making process behind how, when, where, and why we build is no longer working,” said Hamdy, in a statement. “We aren’t just solving a construction problem, we are solving a societal problem. If we are to meet the infrastructure needs of both the developed and developing world, we must improve our decision-making and analysis around the data we have. We are thrilled to have the support of Eniac Ventures as we enter the next phase of our journey.”

A photo of an egg has toppled reality star Kylie Jenner as Instagram’s most-liked post

Instagram has found something it likes more than a Kardashian-Jenner family baby, and it’s an egg.

This weekend, a photo of a plain egg became the most-liked photo on Instagram, the social app owned by Facebook with over one billion users that’s reflective of internet culture.

The photo, which you can see below in its full glory, currently has more than 23 million likes at the time of writing. That has seen it surpass a February 18 photo from Kylie Jenner — the sister of Kim Kardashian and a reality TV star in her own right — which announced the birth of her baby with rapper Travis Scott and has 18.2 million likes.

Unlike Jenner, who has 21 million Instagram followers, the egg account — “world_record_egg” — is a newcomer that seems to have been created in early January. Nothing is known of its ownership, although it now has 2.4 million followers which could — and I can’t believe I’m writing this… — make it an influencer account.

While much can be said about Jenner’s rise to fame, she’s a pretty successful entrepreneur. Her two-year-old ‘Kylie Cosmetics’ brand is estimated to gross over $600 million in annual revenue. While it is funny that a photo of an egg can take the record on Instagram there might be more to it. Jenner’s company trades on her brand, the egg could be a rejection or protest of today’s reality TV culture… which is best embodied by the Kardashians and, in particular, Kylie Jenner. That certainly seems the case looking at the splurge in new and egg-related comments on Jenner’s birth post from last year.

Maybe that’s wishful thinking and this is just another internet phenomenon that can’t be explained. It could simply be a joke that blew up, but don’t discount the potential that this is a stunt from a company launching a new product or wanting to make a splash.

Showing that she might have a sense of humor, 21-year-old Jenner acknowledged the new record in a video of her smashing an egg.

View this post on Instagram

Take that little egg

A post shared by Kylie (@kyliejenner) on

This is the second social media record set this year after Twitter got a new most-retweeted tweet — however, the roles were very much different.

Yusaku Maezawa, a Japanese billionaire who is paying Elon Musk’s SpaceX for a trip to the moon, saw a tweet that offered nearly $1 million in prize money for retweets surpass a true internet phenomenon, U.S. teen Carter Wilkerson. Back in April 2017, Wilkerson took to Twitter to plead for free chicken nuggets; his original tweet now has around 3.6 million retweets.

How open source software took over the world

It was just 5 years ago that there was an ample dose of skepticism from investors about the viability of open source as a business model. The common thesis was that Redhat was a snowflake and that no other open source company would be significant in the software universe.

Fast forward to today and we’ve witnessed the growing excitement in the space: Redhat is being acquired by IBM for $32 billion (3x times its market cap from 2014); Mulesoft was acquired after going public for $6.5 billion; MongoDB is now worth north of $4 billion; Elastic’s IPO now values the company at $6 billion; and, through the merger of Cloudera and Hortonworks, a new company with a market cap north of $4 billion will emerge. In addition, there’s a growing cohort of impressive OSS companies working their way through the growth stages of their evolution: Confluent, HashiCorp, DataBricks, Kong, Cockroach Labs and many others. Given the relative multiples that Wall Street and private investors are assigning to these open source companies, it seems pretty clear that something special is happening.

So, why did this movement that once represented the bleeding edge of software become the hot place to be? There are a number of fundamental changes that have advanced open source businesses and their prospects in the market.

David Paul Morris/Bloomberg via Getty Images

From Open Source to Open Core to SaaS

The original open source projects were not really businesses, they were revolutions against the unfair profits that closed-source software companies were reaping. Microsoft, Oracle, SAP and others were extracting monopoly-like “rents” for software, which the top developers of the time didn’t believe was world class. So, beginning with the most broadly used components of software – operating systems and databases – progressive developers collaborated, often asynchronously, to author great pieces of software. Everyone could not only see the software in the open, but through a loosely-knit governance model, they added, improved and enhanced it.

The software was originally created by and for developers, which meant that at first it wasn’t the most user-friendly. But it was performant, robust and flexible. These merits gradually percolated across the software world and, over a decade, Linux became the second most popular OS for servers (next to Windows); MySQL mirrored that feat by eating away at Oracle’s dominance.

The first entrepreneurial ventures attempted to capitalize on this adoption by offering “enterprise-grade” support subscriptions for these software distributions. Redhat emerged the winner in the Linux race and MySQL (thecompany) for databases. These businesses had some obvious limitations – it was harder to monetize software with just support services, but the market size for OS’s and databases was so large that, in spite of more challenged business models, sizeable companies could be built.

The successful adoption of Linux and MySQL laid the foundation for the second generation of Open Source companies – the poster children of this generation were Cloudera and Hortonworks. These open source projects and businesses were fundamentally different from the first generation on two dimensions. First, the software was principally developed within an existing company and not by a broad, unaffiliated community (in the case of Hadoop, the software took shape within Yahoo!) . Second, these businesses were based on the model that only parts of software in the project were licensed for free, so they could charge customers for use of some of the software under a commercial license. The commercial aspects were specifically built for enterprise production use and thus easier to monetize. These companies, therefore, had the ability to capture more revenue even if the market for their product didn’t have quite as much appeal as operating systems and databases.

However, there were downsides to this second generation model of open source business. The first was that no company singularly held ‘moral authority’ over the software – and therefore the contenders competed for profits by offering increasing parts of their software for free. Second, these companies often balkanized the evolution of the software in an attempt to differentiate themselves. To make matters more difficult, these businesses were not built with a cloud service in mind. Therefore, cloud providers were able to use the open source software to create SaaS businesses of the same software base. Amazon’s EMR is a great example of this.

The latest evolution came when entrepreneurial developers grasped the business model challenges existent in the first two generations – Gen 1 and Gen 2 – of open source companies, and evolved the projects with two important elements. The first is that the open source software is now developed largely within the confines of businesses. Often, more than 90% of the lines of code in these projects are written by the employees of the company that commercialized the software. Second, these businesses offer their own software as a cloud service from very early on. In a sense, these are Open Core / Cloud service hybrid businesses with multiple pathways to monetize their product. By offering the products as SaaS, these businesses can interweave open source software with commercial software so customers no longer have to worry about which license they should be taking. Companies like Elastic, Mongo, and Confluent with services like Elastic Cloud, Confluent Cloud, and MongoDB Atlas are examples of this Gen 3.  The implications of this evolution are that open source software companies now have the opportunity to become the dominant business model for software infrastructure.

The Role of the Community

While the products of these Gen 3 companies are definitely more tightly controlled by the host companies, the open source community still plays a pivotal role in the creation and development of the open source projects. For one, the community still discovers the most innovative and relevant projects. They star the projects on Github, download the software in order to try it, and evangelize what they perceive to be the better project so that others can benefit from great software. Much like how a good blog post or a tweet spreads virally, great open source software leverages network effects. It is the community that is the source of promotion for that virality.

The community also ends up effectively being the “product manager” for these projects. It asks for enhancements and improvements; it points out the shortcomings of the software. The feature requests are not in a product requirements document, but on Github, comments threads and Hacker News. And, if an open source project diligently responds to the community, it will shape itself to the features and capabilities that developers want.

The community also acts as the QA department for open source software. It will identify bugs and shortcomings in the software; test 0.x versions diligently; and give the companies feedback on what is working or what is not.  The community will also reward great software with positive feedback, which will encourage broader use.

What has changed though, is that the community is not as involved as it used to be in the actual coding of the software projects. While that is a drawback relative to Gen 1 and Gen 2 companies, it is also one of the inevitable realities of the evolving business model.

Linus Torvalds was the designer of the open-source operating system Linux.

Rise of the Developer

It is also important to realize the increasing importance of the developer for these open source projects. The traditional go-to-market model of closed source software targeted IT as the purchasing center of software. While IT still plays a role, the real customers of open source are the developers who often discover the software, and then download and integrate it into the prototype versions of the projects that they are working on. Once “infected”by open source software, these projects work their way through the development cycles of organizations from design, to prototyping, to development, to integration and testing, to staging, and finally to production. By the time the open source software gets to production it is rarely, if ever, displaced. Fundamentally, the software is never “sold”; it is adopted by the developers who appreciate the software more because they can see it and use it themselves rather than being subject to it based on executive decisions.

In other words, open source software permeates itself through the true experts, and makes the selection process much more grassroots than it has ever been historically. The developers basically vote with their feet. This is in stark contrast to how software has traditionally been sold.

Virtues of the Open Source Business Model

The resulting business model of an open source company looks quite different than a traditional software business. First of all, the revenue line is different. Side-by-side, a closed source software company will generally be able to charge more per unit than an open source company. Even today, customers do have some level of resistance to paying a high price per unit for software that is theoretically “free.” But, even though open source software is lower cost per unit, it makes up the total market size by leveraging the elasticity in the market. When something is cheaper, more people buy it. That’s why open source companies have such massive and rapid adoption when they achieve product-market fit.

Another great advantage of open source companies is their far more efficient and viral go-to-market motion. The first and most obvious benefit is that a user is already a “customer” before she even pays for it. Because so much of the initial adoption of open source software comes from developers organically downloading and using the software, the companies themselves can often bypass both the marketing pitch and the proof-of-concept stage of the sales cycle. The sales pitch is more along the lines of, “you already use 500 instances of our software in your environment, wouldn’t you like to upgrade to the enterprise edition and get these additional features?”  This translates to much shorter sales cycles, the need for far fewer sales engineers per account executive, and much quicker payback periods of the cost of selling. In fact, in an ideal situation, open source companies can operate with favorable Account Executives to Systems Engineer ratios and can go from sales qualified lead (SQL) to closed sales within one quarter.

This virality allows for open source software businesses to be far more efficient than traditional software businesses from a cash consumption basis. Some of the best open source companies have been able to grow their business at triple-digit growth rates well into their life while  maintaining moderate of burn rates of cash. This is hard to imagine in a traditional software company. Needless to say, less cash consumption equals less dilution for the founders.

Photo courtesy of Getty Images

Open Source to Freemium

One last aspect of the changing open source business that is worth elaborating on is the gradual movement from true open source to community-assisted freemium. As mentioned above, the early open source projects leveraged the community as key contributors to the software base. In addition, even for slight elements of commercially-licensed software, there was significant pushback from the community. These days the community and the customer base are much more knowledgeable about the open source business model, and there is an appreciation for the fact that open source companies deserve to have a “paywall” so that they can continue to build and innovate.

In fact, from a customer perspective the two value propositions of open source software are that you a) read the code; b) treat it as freemium. The notion of freemium is that you can basically use it for free until it’s deployed in production or in some degree of scale. Companies like Elastic and Cockroach Labs have gone as far as actually open sourcing all their software but applying a commercial license to parts of the software base. The rationale being that real enterprise customers would pay whether the software is open or closed, and they are more incentivized to use commercial software if they can actually read the code. Indeed, there is a risk that someone could read the code, modify it slightly, and fork the distribution. But in developed economies – where much of the rents exist anyway, it’s unlikely that enterprise companies will elect the copycat as a supplier.

A key enabler to this movement has been the more modern software licenses that companies have either originally embraced or migrated to over time. Mongo’s new license, as well as those of Elastic and Cockroach are good examples of these. Unlike the Apache incubated license – which was often the starting point for open source projects a decade ago, these licenses are far more business-friendly and most model open source businesses are adopting them.

The Future

When we originally penned this article on open source four years ago, we aspirationally hoped that we would see the birth of iconic open source companies. At a time where there was only one model – Redhat – we believed that there would be many more. Today, we see a healthy cohort of open source businesses, which is quite exciting. I believe we are just scratching the surface of the kind of iconic companies that we will see emerge from the open source gene pool. From one perspective, these companies valued in the billions are a testament to the power of the model. What is clear is that open source is no longer a fringe approach to software. When top companies around the world are polled, few of them intend to have their core software systems be anything but open source. And if the Fortune 5000 migrate their spend on closed source software to open source, we will see the emergence of a whole new landscape of software companies, with the leaders of this new cohort valued in the tens of billions of dollars.

Clearly, that day is not tomorrow. These open source companies will need to grow and mature and develop their products and organization in the coming decade. But the trend is undeniable and here at Index we’re honored to have been here for the early days of this journey.