How to replatform Endeca rules to Solr
Apr 07, 2020 • 8 min read
Apr 07, 2020 • 8 min read
Blockchain technology essentially emerged with the introduction of Bitcoin in 2009. But it is now known that blockchain offers numerous other innovative technical and business use applications beyond what was imagined by the creators of the world’s first cryptocurrency. While Bitcoin had a fairly defined use case as a financial instrument to store value or make financial transactions, people started to see that the blockchain technology that Bitcoin was based on was potentially very useful for a whole range of other applications including IoT, supply chain, media, and retail.
With the launch of the Ethereum project in 2015, the introduction of smart contracts, improved proof-of-work algorithms, and the realization that cryptocurrency tokens could be used across a whole raft of business uses, interest in blockchains increased even further. This is reflected in financial projections for the industry, with recent market research indicating that the international blockchain market is set to grow rapidly over the next five years. It will reach USD $14 billion by 2022, increasing at a Compound Annual Growth Rate (CAGR) of 42.8%.
The blockchain industry is largely being driven by numerous new startups that are building cutting-edge technology services on custom or open source blockchain platforms. These new tech companies are also bringing new ideas and ways of doing business, with everything from novel software solutions to methods raising capital. Unlike the traditional technology sector, where most capital is raised via venture capital or IPOs, many of these companies are attracting funding from a much wider source: the public (via Initial Coin Offerings (ICOs)). These are not shares in the company itself, but cryptocurrency coins or tokens that provide the user with various utility cases, or the ability to hold or sell the token with the hope that its value will increase in the future.
These new blockchains aim to challenge the existing approaches being used across a whole range of industries. This includes the retail sector, where there is a large scope of efficiency improvements and cost reductions associated with blockchain technologies. These optimizations incorporate peer-to-peer supply chain, anti-counterfeiting solution for luxury, and unique items to running loyalty programs. Let’s now look into the essential features of blockchain technologies that provide great potential value to retailers, and how blockchain will transform loyalty backend solutions.
Loyalty or reward programs are one of the key customer engagement and retention strategies that merchants can use to remain competitive in the modern retail environment. However, existing loyalty program models provide many challenges and pain points for retailers. These include the costs associated with establishing, operating, and securing loyalty points programs, low redeem rate of earned loyalty points, customers finding the signup process overly complex, and the general overabundance of loyalty programs in the market. In the US alone, consumers hold more than 3.3 billion memberships in customer loyalty programs.
While these issues with loyalty programs can be viewed by retail companies as simply business cases that need to be solved, there is also an increasingly popular point of view that many of these problems are fundamentally technology challenges. This is especially the case when considering the best ways to operate and secure loyalty programs, given the vital need to secure the data associated with the programs from hacker attacks or altering account balances. This is no trivial matter considering the sheer volume of memberships in existence, and the fact that many loyalty programs, credit card, or frequent flyer programs can easily see individuals with loyalty account balances worth tens of thousands of dollars.
One of the popular business approaches to addressing the low points redeem rate is cooperating with other businesses to offer interchangeability of the reward points within the joint scope of services. While many such programs have been launched, their implementation and operational complexity is high. In particular, they require establishing the B2B communications needed to maintain the accounting aspect of the loyalty points, where each party has to audit transactions and secure data from fraudulent attacks. This is one of the reasons that establishing and operating joint loyalty programs can quickly become a complex and expensive effort.
This is where blockchains really come into play. The loyalty program is considered one of the primary business use cases to consume and adapt the technology in the retail industry, along with supply chain and transparency of product history. Some researchers of the topic highlight a high degree of similarity between cryptocurrency and loyalty points based on the logical chain: Bitcoin -> digital currency -> digital tokens -> loyalty points.
Blockchains are considered to be an excellent solution for running network loyalty programs because they address many of the technical challenges. Some of these solutions include creating a shared transaction ledger, implementing interchanging rules based on "smart contracts", technology-based fraud and cheating protection, and shared scalable infrastructure (the blockchain network itself).
However, despite the technology already being developed to address such loyalty program pain points, not just any blockchain solution can be used to address these types of issues. A comprehensive study conducted in 2017 that looked into leading retailers in Europe and North America identified these key technical requirements to a blockchain to make it compliant with real world needs:
In this regard, despite Bitcoin being by far the most popular implementation, which has proven its security and reliability over the years, it does not meet these requirements. This is because it is a public permissionless distributed ledger with many limitations including scalability, limited size of data blocks, and reliance on the energy intensive. It is therefore expensive to run the Proof of Work consensus algorithm.
The most relevant solutions available in the market today are Hyperledger-based implementations like Hyperledger Fabric or Hyperledger Sawtooth. Hyperledger Fabric is an open-source blockchain solution supported by IBM, which offers a pluggable framework for running a blockchain network in a private and permissioned manner. Intel has also recently launched its own open-source solution within the Hyperledger project called Sawtooth. Sawtooth includes some advanced features like integration with hardware security solutions from Intel, customizable transaction processors, and support of a wide selection of languages for implementing smart contracts.
The Ethereum core development team also invests a great deal of effort into making their popular public network compliant with business needs. This has been achieved via the introduction of private network support, and a move towards the less expensive proof-of-stake (PoS) consensus mechanism.
Below is a brief comparison of how principal business features are currently implemented in three considered solutions:
|Feature||Hyperledger Fabric||Hyperledger Sawtooth||Ethereum|
|Privacy and Security||Private||Private, can be public||Public, can be private|
|Consensus mechanism||Pluggable: Kafka FT, BFT||Pluggable: Proof of Elapsed Time (PoET); Quorum Voting||Proof of work (PoW), Proof of stake (PoS) (coming soon)|
|Scalability||Segregates node roles for transaction execution (peers) and building new blocks (orderers);
low-latency (2 sec) and high-throughout (> 10k tx / sec) with BFT and small number of nodes (< 20)
|Parallel transaction execution||PoW gives low transaction throughput (< 100 tx / sec), the latency of 12 sec, good for large number of nodes|
|Scripting (smart contracts)||Go, Java (coming soon); runs in isolated Docker environment||Python, Solidity, JS, Go||Solidity|
|Primary contributor||IBM||Intel||Core developer group|
There are multiple blockchain-based solutions for loyalty programs that have recently appeared in the market including Loyyal, Chain of points, LoyalCoin, and BitRewards, but no one dominating player has yet emerged. Following the expectations of the retailing market, the implementations are mostly based on private blockchains. However, some are custom built (like Chain of points), or built on an open-source platform Hyperledger Fabric (Loyyal and Wipro).
To understand what makes Hyperledger Fabric the most adopted open-source technology in the retail industry, let’s look at the technical features it offers to address the key requirements of retailers.
Privacy and Security: The transactions ledger is stored on nodes in the intranet with manageable physical access. In addition, Fabric divides the ledger into channels with manageable distribution among nodes in the network, which gives control of where data is physically stored and processed. For transactions with sensitive data, the payload can be stored in an encrypted form.
Permissioned access: All nodes in the Hyperledger Fabric cluster are authenticated by X.509 certificates managed by the membership service provider. In this way, access to transaction data for storing and processing is granted to authorized nodes only.
Scalability and performance: Hyperledger Fabric provides support of the pluggable consensus mechanism. This includes some implementations being available out of the box, such as the Kafka-based fault-tolerant (FT) implementation, and an implementation of the Byzantine fault tolerance (BFT). Both mechanisms allow processing of transactions with low-latency (2 sec) and high-throughput (> 10k tx / sec). However, Kafka FT does not give protection against malicious nodes while BFT does. The trade-off is limited horizontal scalability of up to 20 nodes per cluster to meet the performance metrics stated above. Additional scalability of the Fabric network is achieved by the segregation of node roles into “peers” (responsible for transaction execution) and “orderers” (provide consensus guarantees and produce new blocks in the chain).
Automation of business use cases: Hyperledger Fabric allows executing pieces of business logic as part of the transaction (smart contracts). Currently, it supports Turing-complete smart contracts written on Golang, but it is expected to support Java soon. The code is executed in an isolated Docker environment, which gives additional control over security.
Previously, we have shown how well Hyperledger Fabric fits the demand from the retail industry in general. However, it also serves the needs of the loyalty domain in particular. At a high level, the solution based on Hyperledger Fabric consists of a retailer-specific loyalty application hosted in their infrastructure independently, and a shared blockchain application running in a virtual private space (Fig. 1). The loyalty application interacts with the blockchain via an API, which is exposed to the private network without access from the public Internet. Customers get access to reward points through any retailer’s UI, so customer experience is not directly impacted by the underlying solution.
At the lower levels of detail, the solution can be viewed from the perspective of three layers, where Hyperledger Fabric acts as a decentralized storage and processor of transactions. Hyperledger offers a convenient SDK (available in multiple languages), which is integrated into the retailer’s loyalty application and plays the role of a connector between backend layers. Fig. 2 shows how a sample reward points redeem transaction is processed through all components.
The considered design gives an example of how blockchain technology addresses the specific business problem. However, blockchain-based solutions still have their own challenges and limitations. This is primarily connected with the fundamental feature of any decentralized application - the consensus mechanism. Its function is to guarantee a common and unambiguous order of transactions, as well as maintain the integrity and consistency of the distributed ledger across nodes.
From a performance and scalability perspective, the distributed consensus algorithms used in the considered business-oriented technologies can be classified in the following way:
|Proof-of-work (PoW)||Chain-based proof-of-stake (PoS)||Proof-of-elapsed-time (PoET)||Byzantine fault tolerance (BFT)||Kafka fault tolerance (FT)|
|Implementations||Bitcoin, Ethereum||Ethereum||Hyperledger Sawtooth||Hyperledger Fabric, Hyperledger Sawtooth||Hyperledger Fabric|
This comparison gives us an understanding of the fundamental performance, scalability features, and limitations that each blockchain technology has, based on the underlying consensus algorithm. As a result, choice of technology for solving a particular business problem should be made with respect not only to its features and how well it meets the demand, but also to the specifics of its core design trade-offs. In this view, the pluggability and modularity offered by Hyperledger Fabric and other Hyperledger projects becomes highly important, as it gives flexibility to replace or tune key components.
Similar to how blockchain has been demonstrated to address the business use case of loyalty programs in the retail domain, it can also be applied to many other use cases in other industries. As the first use case was Bitcoin, it was natural that blockchain first be considered as a financial instrument. However, today financial institutions like Santander Bank have identified more than 20 use cases for the technology, including currency exchange and peer-to-peer transactions.
Today, blockchain technology offers far more than just a secure and transparent way for storing and managing digital assets, and it will continue to expand well beyond its initial incarnation as a ledger for digital currency. Medici and GrowthPraxis have conducted comprehensive research based on the close monitoring of startups in the space, and are devoted to better understanding the numerous use cases for blockchain technologies.
Here's a report by GrowthPraxis explaining this in further depth https://gomedici.com/blockchain-use-cases-comprehensive-analysis-startups-invoved/.
Broadly speaking, use cases can be separated from their particular business domain and grouped into the following general categories:
Depending on the use case, solutions can benefit from using either public or private blockchain networks. Public blockchains like Ethereum offer decentralized public infrastructure with secure and protected ledgers, and are a good candidate for many functions, such as proof of asset ownership, financial exchanges, and voting. However, concerns about transaction latency and sharing of data with the public are still justified. At the same time, private blockchains like Hyperledger Fabric are well suited to running P2P contracts, or managing internal and sensitive data in a trusted network.
It is clear that we are still in the early stages of broader blockchain technology development and adoption, despite the rapid advances that are currently occurring. Many comparisons are made between the current state of the blockchain industry and the early years of the Internet, when people were only beginning to understand the huge potential and influence that the new technology could have on both the public and the way that businesses operate.
Despite still being in a nascent stage, we are already observing rapid growth of interest in blockchain technology from business in general, and the retail industry in particular. This will only continue as the pace of research and innovation in the sector continues to increase transaction throughput, and challenges with scalability, latency and security are overcome.
Today’s open-source blockchain solutions can already be applied to solving real business problems, which is being proven by the large number of up-and-coming startups that are solely focused on implementing them. However, it is worth mentioning that there are still a number of concerns. Scalability, performance limitations of production-ready consensus algorithms, and low adoption rates for some specific business use cases are all problems that need to be fixed. Finally, there remains a lack of clarity in effective approaches to resolving legal issues connected with execution of smart contacts in the decentralized blockchain.
If you have any questions, feel free to leave us a comment below.