New York chatbot spreads false information

A recent investigation by The Markup revealed that a chatbot launched by the New York City government to provide information support to business owners has been spreading inaccurate and even misleading information about city policies.

Here's What We Know

According to the report, the chatbot, powered by Microsoft Azure's artificial intelligence, provides false information, sometimes even breaking the law.

The publication, conducted in conjunction with local non-profit publications Documented and The City, found a number of inaccuracies in the chatbot's answers to questions about housing policy, employee rights, and other significant topics.

Mayor Adams unveiled the chatbot in October as part of the My City portal, which was introduced in March 2023 as the city's main source of services. However, it turned out that this was only a pilot project, with a warning on the website that sometimes the information could be inaccurate or even harmful.

During The Markup's testing, the chatbot provided incorrect information in response to various questions, including questions about cashless shops and employment rules. Housing policy experts describe the chatbot as "dangerously inaccurate" at worst.

Leslie Brown, a spokesperson for the New York City Office of Technology and Innovation, said that the chatbot has already provided thousands of people with timely, accurate answers. However, he admitted that their team will continue to work on updating the system to provide better support for small businesses in the city.

Source: The Markup