Product recognition software enhances item-level oversight

April 5, 2011

(Editor's Note: Following is an expanded version of a feature article that ran in the May 2011 edition of Automatic Merchandiser Magazine. The article below ran in Automatic Merchandiser up until the subhead, "Product Recognition Solutions: Vending Operator Perspective.")

For years vending operators have struggled to find a more exacting method to coordinate products in a machine with reported sales data. The ability to link DEX data to sales information is often based on a pre-determined planogram or product map; both of which rely on products being placed in a vending machine with strict adherence to the scheme. Unfortunately, to date there has been no easy way to prove that strategic product placements were completed as planned, thereby complicating marketing, rebate, and sales analyses.

Similarly, product manufacturers striving to gain strategic advantage may encourage or influence vending operators through incentive programs oriented toward shelf placement and/or number of facings. While operators wrestle with menu mix, manufacturers tend to seek commitments dictating the number of product spirals dedicated to their products among various rows and columns. Like adherence to a product map, assuring manufacturers that such agreements are strictly followed can be highly challenging.

Vending operators tend to price products on the same shelf at identical prices thereby minimizing operator anguish. But the opportunity to present a broader array of items, with differing prices, is becoming more appealing as consumers and operators recognize premium products often carry correspondingly higher prices.

The problem with traditional vending is that selling prices need to be determined and posted or displayed in static locations beneath the product facing. An error in product placement or a mixture of dissimilar products in a single column can result in diminished contribution margin and/or lost sales; especially when the product is incorrectly priced.

Consumers have a bad perception when products are over-priced and conversely operators are negatively impacted when products are sold under-priced. A solution to this problem would be the ability to set a price for each product and to have that price displayed and available when the product is in the front spiral position (next available for sale).

The need to display nutritional, and often ingredient, data is becoming of significant importance to the vending industry. Given that legislation intended to assist the consumer in making an informed product choice will soon be mandated, vending operators will not only need to know the exact identification of a product in each facing, but will also need a way to display its nutritional content.

A consumer interested in purchasing a breakfast pastry, for example, may use the machine capabilities to review the item's nutritional data. But if the response to the query leads to the machine displaying the nutritional panel for a salty snack, much confusion may ensue.

Is the actual item presented for sale the one linked to the nutrient file? It appears likely that inaccuracy in matching product to nutrient description may have unintended sales consequences. Knowing which product appears where in the machine at any point in time can be critical to operational success.

The above concerns, as well as other issues, are addressed and resolved through product recognition (PR) technology developed by Vendors Exchange International, Inc. This innovative high-tech solution is designed to specifically determine the content of each spiral in real time, reference its price, and provide the opportunity to link to nutritional and key ingredient data.

What Is Product Recognition In Vending?

 Product recognition in vending requires a photographic mechanism to match the image of a product in a spiral facing to a stored database of products to determine its stock keeping unit (SKU) code number and selling price. When integrated with nutritional information, product ingredient and nutrient content can also be accessed through VEII's MIND technology. While product recognition technology involves several important concepts and features designed to enhance effectiveness, the following five concepts are intended to provide an overview of the product recognition application; there are several additional characteristics and capabilities mentioned throughout this article.

Product Recognition – the object of product recognition is matching product images to stored photographs to confirm product identity. Matches lead to access to SKU, product pricing, and optional linkage to nutritional database information (when integrated with MIND software).

Digital Pricing – given product identification within the recognition system database, the price of an item can be accessed and posted as a digital price; the item's price is based on its stored price, not the item's location in the vending machine.

Diagnostic Tool – the camera component of product recognition can be used to check machine status, operational status, and overall appearance. An empty facing, hung-up product spiral, or an unrecognized product are conditions that may traditionally result in machine downtime can now be remotely diagnosed and resolved by remotely positioning the camera over the problem spiral and in real-time, remotely operating the spiral to visually see if the problem can be corrected.

Bijection – bijection is a process in which mathematical algorithms (formulas) are used to match a product image to stored file content. In the case of vending, the matching can be accomplished regardless of the orientation of the product package in the machine spiral. Bijection is designed to decipher physical attributes against database imagery (one-to-one recognition) and is similar to biometric measurement, except it does not include human characteristics or attributes.

MIND Software – a product recognition application can be optionally applied with the standalone application from VEII labeled Making Informed Nutritional Decisions (MIND). The MIND involves linking manufacturer supplied nutritional and ingredient information to vendible product offerings. When product recognition is integrated with nutrient and key ingredient data of the MIND, data files can be displayed on the touchscreen of the application.

How It Works

 In preparation for product recognition, a camera is used to capture a series of product images in varying orientations (e.g. right-side up, upside down, forward, backward) to create the highest probability of positive identification. This variance in angular views, combined with acknowledgement of font style, font size, and color, lead to minimal misevaluations or failed reads. The degree of detection effectiveness is an important concern whenever a bijection evaluation (one-to-one matching) is performed.

How Effective Is It?

 From a comparison perspective, snack products are considered easier to identify than bottled beverages that can rotate up to 360 degrees thereby necessitating more database images for identification matching. Although with snack items there may be some wrinkling or unreadable text, most packaging is of a bag or loose wrapping that places the item forward, backward, right-side up, or upside down thereby requiring relatively fewer database images to test for identity than the potential challenges of a bottled item.

Packaging that can be angled; uneven, wrinkled, or condensed might lead to false readings. While it may be impossible for the software algorithm to correctly identify every product in each spiral 100 percent of the time, the goal of product recognition software however is to achieve a correct recognition rate of at least 95 percent. When the packaging of two products may be so similar that differentiation may be based only on a small icon or graphic logo located somewhere off-center of the wrapper facing, distinguishing the correct product presents a complex recognition challenge.

For example, consider a Hershey Milk Chocolate candy bar and a Hershey Milk Chocolate with Almonds candy bar. The wrappers of the two bars are very similar in color, font, text, and shape. Discriminating between them through the lens of a digital camera, given the items are vertically slotted behind a spiral or other armature, illustrates the possible difficulties in attaining a perfect identification rate.

When such a confounding situation presents itself, product recognition software can default and assume that the current item is identical to the last recognizable item that was in that exact location, or be programmed to return a blank image and default to a pre-programmed price.

Power Of Seven

Product recognition software is composed of seven distinct algorithms that simultaneously diagnose product appearance/physical characteristics (e.g., color, shape, size, position, font, text, tilt orientation) in an attempt to determine a product database match. Once the recognized product is identified and located in the product recognition database, the product's SKU and price are accessed and may be displayed on the machine's optional MIND touchscreen display and/or on its digital price label.

The ability to accurately distinguish fine points on packaging is the goal of product recognition software. A product that is not identified can be assumed to be the same as the product that preceded it in the spiral or a "not available" message can be displayed. In either case, the machine can continue to function without interruption.

Alternatively, when the product recognition software concludes that a candidate product is unrecognizable, the software can shut down the mystery spiral and render it out of order; this would likely be a last choice situation. A mid-90 percent recognition rate for a bijection measurement application is considered excellent and this software is expected to routinely perform at that level.

What Is Bijection Measurement?

The word bijection means to have equivalent mathematical algorithmic outcomes so that an image can be matched to a stored item value. This is similar to a biometric comparison with the exception being that there are no human attributes or physical characteristics connected with the target.

To adequately assess the identity of a product, given multiple orientation and angular photographs, the array of images must be present in the vending machine's product recognition database. By having a collection of photos of the product, recognition of an improperly stocked or scattered product placement should not hinder recognition efforts.

The orientation of a product in a spiral cannot be guaranteed and the collection of stored database images must be sufficient to enable matching the correct product with its SKU and pricing data as well as optional access to nutritional content information should the MIND application also be implemented.

Maximize Space to Sales

 A unique feature of product recognition is the ability to vend identical products from multiple spirals even when one spiral enjoys the most advantageous machine placement and the other a less desirable machine shelf space.

For example, an operator may decide to offer more than one column of a particular product to avoid frequent re-stocking or stocking out of a high-demand product. When the consumer purchases the item, regardless of which spiral the consumer selects, the actual location of the dispensed product may be different than the one selected.

Given that the second column of the product is located in a less desirable space in the machine (e.g., bottom row) then the product recognition software can be programmed to balance out the product inventory by vending from the least desirable placement first (until it is empty) or dispensing the product alternatively between the two columns to avoid a sold out situation for either spiral. In either case, the product remains visible in the more desirable machine location while the machine remains balanced with a robust appearance.

Ensure Proper Price Points

 There is great value in having confidence that products will be properly priced by the product recognition software shortly after the product appears in a spiral facing (i.e., presented to the customer for purchase). This information is critical to a successful sale and a machine equipped with a digital pricing tray or LCD display screen can project the product price immediately.

In essence, the software can change the displayed price from the last item that appeared in the spiral location (which may have had a different price) to the new price. This feature enables a vending operator to deviate from the common industry practice of keeping similarly priced products in the same row or column or throughout the entire machine.

A Tool For Alerts/Alarms

 Situations such as product delivery failure, empty facings, and related problems can be resolved through an external notification mechanism contained in the product recognition application. A text message sent to a service manager may result in the manager correcting the error through remote camera/machine interface.

The vending machine camera can be directly positioned over a non-vended product, for example, and the machine then instructed to rotate the spiral in an attempt to dispense the product in question (assured delivery). This feature is capable of replacing guaranteed delivery or delivery sensor technologies available in the marketplace (e.g. surevend, ivend, golden eye, etc.).

Field Testing Begins

 Vendors Exchange has started live field testing of its product recognition software and soon will be able to evaluate the accuracy, speed, and integrity of system components. The percent of correct product identifications, proper linkage to stock keeping unit, and correlation of price and nutrient data all will have a profound impact on the potential of the technology.

Product Recognition Solutions: Vending Operator Perspective

Following are benefits from an operator perspective:

  • Product Assurance – more accurate individual product sales data and confidence that a product was in fact where it was intended to be will increase revenue. The ability to instantly return a "live" product map from the machine will assure the vending operator of the product mix available now in the machine. The camera can be remotely controlled, if desired, and can actually vend a product if need be. This can be especially beneficial if an operator is relayed an automated alarm that a column is suspected of being out of service (for example, an item is hung up or if no sales from a specific spiral are noticed after analyzing a machine's sales data).

  • Monitored Facings – the number of facings in a machine can be controlled through product recognition designed to alert the operator when a desired number of facings is exceeded. For example, perhaps management desires that no more than two facings of any item is desirable. When this number is exceeded, the machine can issue a notification to the operator to correct the discrepancy. In essence the camera provides a real time plan-o-gram.

  • Fill Confirmation – when a vending machine is restocked and the door closed, the product recognition camera is capable of performing a comprehensive photo collage of all machine spirals. This view can be used to confirm adherence to a pre-determined planogram and sets the starting position for the ensuing sequence of transactions.

  • Digital Pricing – as a product is dispensed and the next item in the column is moved to the front, the product recognition software is used to identify the product, and where appropriate ensure that its correct price is displayed on the optional new digital price label. This enables a wider variety of price points to be offered for sale and will not force all products of the same price to be contained to a select set of spirals.

  • Pre-Kitting Integrity– using remote machine monitoring data to determine which products require replenishment can be significantly enhanced when product recognition is used to explicitly identify product velocity in a vending machine.

  • Consumer Goodwill – information on the correct product can be displayed with confidence as manufacturer nutritional information is displayed based on an accurate recognition of each product in each spiral. It should be noted that selling products at a fair price (as an alternative to line pricing or over-pricing) will likely contribute to consumer goodwill.

  • Diagnostic Tool – camera can be used to review operations as well as to remotely direct in the case of a non-vend or machine malfunction. Also, the product recognition camera can double as a drop sensor capable of confirming product delivery by the machine and act as a trigger to send an alarm if a specific column fails to vend.

Product Recognition Solutions: Product Manufacturer Perspective

Following are benefits from a product manufacturer perspective:

  • Compliance Concerns – assume an agreement between a product manufacturer and vending operator to provide a specific number of facings, and even specific spiral location, compliance is important. How can the manufacturer, and operator for that matter, be sure that the proper number of facings has been provided? Product recognition can be used to capture the product offerings in all spirals on demand or as sales occur. This type of documentation can be useful in rebate programming, velocity of sales analysis, and contribution margin analysis. Suspicions are relaxed with real time, accurate photographic evidence.
  • Value Concerns – proof of sales depends on proper machine filling and replenishment. A careless or sloppy fulfillment will lead to a lack of value for both the manufacturer and operator as well as the consumer. The central issue being that vending operators tend to price all items in the same row (shelf line pricing) at the same price to minimize the chance that a route driver might inadvertently place a higher cost item in a slot with a selling price that will result in a smaller margin or even at a loss. To avoid this situation, operators tend to price all items on the same row at the same selling price. For both the consumer and the product manufacturer, this may have unintended consequences. While the practice of shelf line pricing is designed to avoid selling items at a price that is less than preferred, it has the potential when lower cost items are sold at higher prices, to present an overpriced menu mix when in reality there may only be an item or two that are inappropriately priced. This could eventually lead to poor machine velocity and therefore insufficient sales.

Product Recognition Solutions: Consumer Perspective

Following are benefits from a product consumer perspective:

  • Accurate Product Data – important consumer data (manufacturer's nutritional information) can be displayed on a multi-touch screen as selected by the consumer by identifying the candidate product by spiral location, the consumer directs the machine to display the nutritional composition of the vendible product. Product display also likely will meet the governmental requirements for nutritional data in vending.
  • Intelligent Product Selection -- smarter purchase decisions can result from better informed product choices. The displayed data can be used as a basis for eventual product selection by the consumer.
  • Value Proposition - better pricing schemes provide consumers with better value.

 Product Recognition -- Required Hardware

Following are required hardware to implement product recognition software.

  • Digital Camera – the device used in product recognition is an electronic board mounted camera capable of capturing color images in even low light conditions. The motor-drive speed of the camera lens enables rapid taking of shot sequences to confirm product dispensing (sure vend) and/or hang-ups or malfunctions. Once image is linked to stored image, the product's stock keeping unit (SKU) is used to provide inventory, pricing, and related information.
  • Revision Door -- retrofits are not expected to be readily available – a VEII custom designed vending machine door with camera-ready navigational apparatus built into the door to govern precise camera movement. Revision doors are available in glass front and (soon) with an LCD video monitor front. Universal Control Board (UCB) – the Vendors Exchange plug and play control board is designed to fit most snack vending machines, is equipped with DEX and MDB interfaces as well as an SD card slot for simplified updating of programming, pricing, and other data. The UCB also enables peripheral device connectivity including an LCD touchscreen for the MIND, a motion sensor for LED lighting control, cellular remote communication channel for machine alerts, and a security camera for producing time/date stamped images. The UCB can also be configured with a guaranteed delivery sensor.

Product Recognition -- Optional Hardware

Following are optional hardware:

  • LCD Monitor (small multi-touch screen) – monitor is not required for product recognition as images are stored internally. An LCD touchscreen can enable the linking of the MIND nutritional and key ingredient data to be displayed.
  • Digital Pricing Labels (on the tray) – used in combination with item data storage.
  • Video Monitor (Closed) Door – large format screen requiring UCB and camera-ready axis pathways (must have connectivity to digital content management software and media distribution).

Product Recognition -- Required Software

Following are required software for installing product recognition software:

  • Vendors Exchange Product Recognition Software – in order for a product recognition application to be able to capture and match images registered by the digital camera to stored images in the associated database files. A real-time application that can also be used to troubleshoot and/or resolve a variety of machine alerts and alarms as well as the capability for selective on-demand remote control operations.
  • Product Recognition Database Management – content restricted to stored product orientation images in a manner designed to facilitate rapid and accurate evaluations of real-time captured images.

Product Recognition -- Optional Software

Following is optional software:

  • MIND – Nutritional/Ingredient database filing application
  • Nutritional Database (nutritionaldatabase.org)
  • Display Builder (for MIND screen design)
  • Digital Pricing (item price linked to item, not shelf or line)

What is MIND?

From a consumer perspective, knowing the nutritional information prior to a purchase may be a critical factor in item selection. The MIND (Make Informed Nutritional Decisions) software enables the consumer to either navigate through a series of attribute screens (category screens) or enter a product's row and column coordinates on a digital keyboard to reveal a product's nutritional, and sometimes dominant ingredient or allergen content. When attributes are used, a series of lead-through screens provide the consumer with product information to enable a more informed purchase decision. The same outcome results when the consumer accesses the data via spiral reference index. Each of these processes requires the consumer to interact with the display screen to retrieve the correct item information. With product recognition, available items are identified automatically by the machine-based software and can be displayed akin to the manual search procedures (if desired).

The MIND enables a vending operator to download the nutritional information of products being placed in a vending machine from a website (NutritionDatabase.org.) Once the nutritional data is captured, it is stored on an SD card connected to the vending machine control board. The nutritional information, as well as a picture of the product, can then be displayed on the MIND monitor. The nutritional database contains an extensive listing of product information as provided by its manufacturer. On the vending machine a video monitor (screen display) is linked to the board is then able to display the manufacturer's nutritional product information, as well as playing promotional videos, including commercials and advertisements. In essence, the database containing the product's information is populated based on identification of the products being sold through the machine.

The nutritional database contains information provided by the product manufacturer (commercial or private label) who remains responsible for maintaining current content. The database can be accessed by a vending operator, who in turn, selects products to be sold in the vending machine thereby creating a customized data set for each machine configuration. It is important to note that the nutritional database used in association with the MIND is established through the direct contribution of product manufacturers. For operators, the expense involved is the cost of the MIND touchscreen display and related mounting hardware. The operator then maintains the information on the screen through the use of Display Builder software, provided at no cost, and the use of a non-volatile Micro Secure Digital (SD) card for file storage.

While the MIND can be installed on nearly all vending machines (new and legacy machines), the same is not true for product recognition which for the time being requires a Revision Door from VEII to operate. It is the apparatus in the door that provides the pathways for movement of the recognition camera through exacting, precise patterns (X and Y axis coordinates).

Additional Benefits And Considerations

Following are other considerations and possible benefits of product recognition software. 

  • Positioning – realizing that there are prime spiral locations in vending machines is similar to recognizing advantageous shelf locations in a convenience store or supermarket. By knowing the prime locations within the vending machine space, this real estate can be strategically offered to product manufacturers. Product recognition software can be used to create a photo collage that proves product placement adhered to negotiated spiral allocations.
  • Location Lift – the vending operator can correlate item sales based on where in the machine an item is placed (glass front) against its velocity of sales. Assigning a product to a favorable spiral location coupled with a reasonable selling price will contribute to overall revenue success through sales lift. When an item is found to be slow moving, it can be moved to a more better position within the machine, re-priced or replaced.
  • Account Acquisition / Retention – product recognition software can be a competitive advantage in the negotiation or renegotiation of a vending account or location. Additionally, integration with MIND technology can provide a broader array of competitive advantage.
  • Cleanliness – the camera can be used to photograph the inside of the vending machine and thereby become an effective service tool. Important to vending is the cleanliness and general upkeep of machinery and product display.
  • Alternate Streams of Revenue – the MIND can be used in combination with product recognition software to display nutritional and ingredient information in addition to SKU and product pricing data.
  • Flash Videos – if an LCD screen is installed on the vending machine, than there is the possibility of playing commercial advertisement videos of in-machine products (either as part of the MIND application or independently) and for presenting flash video promotions from nearby retail stores, movie theatres, restaurants or other surrounding businesses. 

One of the most impressive innovative vending technologies is product recognition software. The ability to identify each product in every spiral can help vending operators be more efficient, product manufacturers get more effective, and render consumers better informed. When linked to nutritional data, product recognition evolves into an end-to-end consumer interface that enhances the vending experience. As Frank Guzzone, business developing manager of strategy and innovation for vending and OCS at Kraft Foods explained, "Combining the MIND with a product recognition camera represents a tremendous leap forward in vending technology." This step could help launch the next technology platform necessary to revitalize the vending industry.