Big Data, Analysis and Business Intelligence

Big Data, Analysis and Business Intelligence

big dataThere is a lot of talk among IT professionals of “big data”, and discussions at many business conference tables center on how the organization might find greater benefit and advantage from the intelligence buried in the business systems and information.  It is a two-part problem, where the collection and the analysis each play essential roles in developing real business intelligence.

“So what’s getting ubiquitous and cheap? Data.

And what is complementary to data? Analysis. ..”

Hal Varian, Chief Economist at Google and emeritus professor at the University of California, Berkeley
“Hal Varian Answers Your Questions,” February 25, 2008 (http://www.freakonomics.com/2008/02/25/hal-varian-answers-your-questions/).  BUSINESS INTELLIGENCE AND ANALYTICS: FROM BIG DATA TO BIG IMPACT; MIS Quarterly Vol. 36 No. 4, pp. 1165-1188/December 2012

The information technology and systems in a business support the operation.  Software and computers help people do their jobs, and the information collected in and generated by those systems becomes the foundation for developing business “intelligence”.   Today, businesses must reach beyond their own direct operational support systems and consider the full realm of data to be collected, including IoT sensor data or social media data.

Business intelligence is gained from the analysis of the critical business data – analysis which helps owners and managers make better and more informed decisions which are based on an understanding of the business and market.   Business intelligence was a term popularized in the 1990s, but the key was the analytical component (business analytics), which gained focus in the late 2000s. Today it is big data and big data analytics, where organizations are working with massive data sets not previously even imagined.

“…one of the most significant challenges facing enterprise IT teams today is how to efficiently support and enable the “science” of big data, while providing the confidence and maturity of more traditional (and often better understood) infrastructure services.”

http://www.datacenterknowledge.com/archives/2015/05/26/hadoop-big-data-storage-challenge-overcoming-science-project/

The volume and velocity of information collection is ever-increasing even in the smallest of businesses, creating a great need for tools which can structure and correlate data so that it might render some insight.  Simply storing and managing these huge and growing data sets has become a challenge, and there isn’t one right way.

Once the business has the data, then it must find a way to analyze the data, which generally involves also applying visualization tools. Many IT departments are feeling pressured in the development of new skills and capabilities around data collection and management, yet it is more frequently the business user who provides the analysis and applies visualization tools to the task.

“Data collected by the Aberdeen Group, found that employees in organizations that used visual data discovery were more likely to find the information they need, when they need it. These same companies were able to scale their use of scarce IT skills more effectively.”

http://www.tableau.com/learn/whitepapers/visualization-set-your-analytics-users-free#0vXrkWZbizxyutw

The use of business intelligence and advanced analytics continues to grow in every segment of the market – from small business to enterprise – and plays an increasingly important role in supporting business success.

Until this point, most businesses didn’t have the technology or the data to enable significant quality or business transformation, but the times are changing and deployments of data collection, analysis and visualization software and tools are expanding with it. This is a fundamental aspect of business digital transformation and fuels the next step, where intelligence is applied to conditions revealed in the data and activities are automatically performed guided by that intelligence.

jmbunnyfeetMake Sense?

J

Robots are just automation you didn’t know you needed

tall-tower-vancouver
There is a big trend in software integration these days which involves automation – turning connected systems into free-flowing conduits for data to move intelligently into and out of with ease.  Well, maybe not quite that, but the key is the unattended and intelligent movement of data from one system to another.  People don’t have to get involved in order for the information to flow from one system to another… it just goes by itself.  Like a robot.

Having software and systems connect to one another isn’t new at all, and businesses have for many years recognized the value of being able to have information entered in one system available in another.  Entire ERP frameworks have been created based on this concept of entering data once and using it in many ways.  The trick is when there is more than one system or software product involved.

A simple example might be someone who owns a web store and does their bookkeeping with QuickBooks Premier.  The webstore isn’t running QuickBooks; it is running an e-commerce solution or shopping cart system that allows customers to buy things online.  However, the webstore does create sales orders and charge transactions, and may even have to manage an inventory of saleable items.

Getting the information from the webstore to QuickBooks and vice versa is often a task business owners take on, either by entering the information manually themselves or hiring and employee to do it.  This manual re-entry of information introduces a large potential for errors in the data entered and is time-consuming and costly.

“It was just awful,” said David Clothier, treasurer of the Knoxville, Tenn., company, which operates more than 500 Pilot Flying J truck stops nationwide. “There were humans everywhere.”

http://www.wsj.com/articles/the-new-bookkeeper-is-a-robot-1430776272

Rather than having a person re-type the information from one system into another, software-based integration programs may be available to help users map the data and move it from one solution to the other.  Using a software program to transfer the information is much faster and reduces the error rate, increasing the overall value and usefulness of the information.

Automation isn’t the only requirement that makes this all robot-like.  The additional requirement is the intelligence.  If people still have to get directly involved in order for something to happen, then all the happening is still based on human performance. No robots here.

Intelligent integration of information occurs when the systems on both ends are capable of making decisions and acting on them.  For example, a business might use a solution that allows vendors to submit their invoices electronically.  Through a base of rules that match invoices to requests and approvals, the system is able to issue payment and record the transactions automatically and without human intervention, saving hugely on personnel and processing costs.  Robots (the automation solution) wouldn’t make up all the rules, but could follow them repetitively and without question once established.

…software can help businesses operate more effectively. “If you think like a human, there are only certain things you can do. When you think like a robot, many things are possible.”

http://www.wsj.com/articles/the-new-bookkeeper-is-a-robot-1430776272

It isn’t a new paradigm for building businesses – this doing of things a bit smarter than before and leveraging technology to get more done in less time.  The difference is that the pace of change is increasing, giving businesses less time to stand by and rub the chin and consider whether or not this new fangled technology makes sense. Robots, you say? Meet your new part-time bookkeeper.

jmbunnyfeetMake Sense?

J

The Cloud and the Business Desktop

Cloud computing is here – no longer is it considered to be temporary or just a fad.  Even though there are many businesses in the country without access to high quality high-speed Internet connectivity, the levels of investment and revenue surrounding cloud and mobile computing solutions and technologies has proven that mobility and managed service matter to those who are connected. What’s interesting is that the popularity of the cloud and the emergence of cloud-based applications and services haven’t really put much of a dent in the need for the desktop, which remains as the business workhorse and – connected or not – represents the foundation for business productivity and getting work done.

Some years ago, business applications began to emerge in SaaS (software-as-a-service) format, meaning a customer could simply subscribe to an application on the web rather than purchasing and installing software.  This option clearly resounded with many business customers and ushered in an era of online application services oriented specifically toward mobile users. Yet the desktop remains as the place where online solutions meet productivity (export any online data to an Excel spreadsheet recently?) and where accounting and finance connect with the rest of the operation.

Believing too much of the marketing-speak around cloud computing, many business users believe that they can only remotely access business software solutions if they are “cloud” and subscription model applications, and that the desktop products they know and have invested in cannot be available to them in a fully managed online model.  In fact, a large number of the business owners I speak with that actually use hosted desktop services somehow believe that the software they are using is something special and different from that which would be installed to their PCs. The fact is that the software is not different, regardless of what they may think. More often than not, the hosted applications are EXACTLY what the customer had previously installed (or would have installed) to their own computers had they not been working with a hosting provider.  Whether they are hosted or not… the desktop products generally function with all the features and capability designed into them because they are hosted on platforms they were designed to run on (like Microsoft Windows, for example).

Customers of the QuickBooks hosting companies often refer to their systems as “QuickBooks cloud, but not the online one”, not really understanding that it is simply the full desktop application that is being hosted for them.

Regardless of how many online application services emerge, and even if (IF) web-based versions of our favorite word processing and spreadsheet software become as useful as the installed kind, there will still be a need for the desktop if for no other reason than to make it easier to use and work with a variety of solutions at the same time.  Perhaps this is why remote desktop computing and hosted application services are becoming increasingly popular approaches to cloud and managed computing services.  The user benefits from having the feature-rich applications they need and a single place to access them and make them work together (the desktop value proposition), yet is able to have remote and mobile access, comprehensive system management and maintenance, data protection, helpdesk support and affordable monthly payments (the cloud value proposition).  In many ways, application hosting models represent the best of both worlds for the business.

JJoanie Mann Bunny Feet

Make Sense?

Consider how beneficial it would be to businesses who want the advantage of remote desktop and mobile access to applications to be able to run their QuickBooks (feature-rich desktop QuickBooks) and/or other business applications in an anytime, anywhere sort of environment. Businesses can obtain hosting services for QuickBooks Pro, Premier, and Enterprise – allowing organizations to have their QuickBooks financial applications managed, protected, secured, and made available to users all the time and from any location. Some hosting services may also support integrations and extensions for QuickBooks – for both desktop and Web-based applications and services. When the host can provide authorized subscription licensing for Microsoft Office, a business can have a complete, outsourced IT solution and pay only monthly service fees to get it. No installation or system management to worry about: the QuickBooks financials, the productivity, the operational systems and plugged-in applications can all be hosted in the cloud.

Run Your [New, Small, Growing] Business from Anywhere

The office for a small business used to be where all the work got done.  The hub of activity and productivity for a small business, the office was where you could connect with team members and co-workers and generally keep on the same page with what was going on in the business.  Customer orders are taken, those orders are fulfilled, and bills are paid – all from the small business office.  Yet today’s small business isn’t tied to the office location any longer.fishingpoles

Mobility and the cloud now provide businesses with mobile office options that allow users to get their jobs done no matter where they happen to be.  Business moves at a fast pace, and mobility and remote access solutions help companies be more nimble.  Collaborating while on the go and exchanging ideas and concepts quickly helps businesses be more agile and better-able to meet changing customer needs.  Successful small business owners leverage mobility and action to beat the competition.

The cloud and Internet-based computing lets small businesses access and benefit from IT solutions that were previously only available to enterprise organizations.  Better IT means being more competitive, giving smaller businesses a leg up and positioning them among even the largest of competitors. For the business owner, the freedom of being able to manage the entire business from anywhere delivers a freedom and flexibility previously unimagined.

Here are some ways hosted and cloud-based IT can help small businesses overcome everyday business challenges:

Reduce or Eliminate the Need for a Physical Office

Starting a business is tough, and many small business owners decide to use their own homes as a business location rather than forking over a bunch of lease money to a commercial realtor.  Using hosting application services and cloud technologies can help keep team members and co-workers working together, no matter where they are located.  Many businesses are able to get off the ground and operating successfully without ever having an established office.

Work when it Works for You

Remote desktops and hosted applications deliver functionality to users no matter where or when they need to work.  With ready access to everything needed to get the job done, workers are able to be productive even when they’re not at a desk (or even a computer!).  Smartphone and tablet apps can make working from a mobile device highly effective, extending productivity and capability to workers whenever and wherever it is required.

Keep Everyone on the Same Page

When systems are centrally located and accessed, it is easy to keep everyone on the same version, the same edition, and the same page.  No matter where users are located, documents and application data are kept in sync, ensuring that everyone is working on the most current information available.  Mobile access to applications and data keeps information from being distributed to various devices, making revision control easier and providing better protection for valuable business information.

Mobile computing and the cloud make it easy for small businesses to have better IT that enhances productivity and supports growth.  Reducing capital costs and exchanging large technology investments with affordable monthly subscription service gives small businesses the boost they need to implement the solutions and services which will develop and improve collaboration, streamline workflows, and reduce overhead costs while enabling a fast-paced and agile business ready to meet any challenge.

jmbunnyfeetMake Sense?

J

No REST for QuickBooks Desktop Integration Developers

No REST for QuickBooks Desktop Integration Developers

elastic-cloudIntuit, the maker of QuickBooks small business accounting software (among other things), is discontinuing service for the REST API and the Sync Manager on March 1, 2016 [1].  Developers with applications which integrate with the desktop editions of QuickBooks using this method must change their approach right away or risk having their integrations simply stop functioning.  It’s not that Intuit will DO something on March 1st.  Rather, they’ll stop doing something – like handling Sync Manager integrations.

There are a lot of different types of businesses in the world, and each of them produces and consumes a lot of information.   From sales to human resources; from operations to finance – every business generates and manages information to support the various processes which make up the business activities.  Computer systems and software represent the tools businesses use to develop and manage information, and often become foundations for structuring the information which flows through the organization. Just as there may be different people in the business, each with their own responsibilities and job functions, there are likely software applications which are similarly oriented to support different processes within the business.  Integrating or connecting different applications and processes within the business helps the organization be more efficient with information usage, generally increasing the quality of access and reporting throughout the business while at the same time reducing or eliminating redundant data entry and the potential for errors.  Software integrations are a big thing to many businesses, which is why the discontinuation of Intuit’s Sync Manager for QuickBooks Desktop editions is a big deal.

Intuit’s Sync Manager was the big thing just a few short years ago.  Providing developers with a seamless method for accessing QuickBooks company data and passing it to/from web-based and other applications was a boon to the online application model and paved the way for many disk-based integrated solutions to migrate to SaaS offerings instead.  Developers who saw success operating in Intuit’s QuickBooks marketplace as recognized add-ons were encouraged to use Sync Manager so that they would be able to seamlessly market to, subscribe and onboard new users who purchased QuickBooks products. Whether or not the developer participated in Intuit’s application marketplace, the Sync Manager and the REST API provided them with some very important capabilities and supported new methods now recognized as “standards” for development of web-based solutions and services.

The World Wide Web has succeeded in large part because its software architecture has been designed to meet the needs of an Internet-scale distributed hypermedia system. The modern Web architecture emphasizes scalability of component interactions, generality of interfaces, independent deployment of components, and intermediary components to reduce interaction latency, enforce security, and encapsulate legacy systems. http://dl.acm.org/citation.cfm?doid=337180.337228

In order to integrate a solution with QuickBooks desktop products, there are two essential problems to solve.  First, there must be access to the QuickBooks data.  Few products are able to directly access the data in a QuickBooks data file; generally, the QuickBooks program itself is used to ‘broker’ access to the company file. So, developers need a way to work inside of QuickBooks to use it to access the data their applications need.  Second, the data must be transported (via the Internet) to allow for data to come from QuickBooks into another app, or to allow data from the other app to come to QuickBooks.  The REST API and the Sync Manager addressed both of those problems and provided developers with the mechanisms required to facilitate the data integration as well as transport the data.

REST (representational state transfer) is “the software architectural style of the World Wide Web [2]” and represents a standard for creating scalable, distributed system interactions.  Using this method, developers were able to make their online solutions access, read and write data in QuickBooks desktop products because Intuit had first sync’d the data to its servers, so developers needed only to reach the Intuit servers to reach the data.  The Sync Manager provided the transport, carrying the data to/from the desktop installation where the Sync Manager service was running.  And, because the Sync Manager was basically built-in to QuickBooks, there was no additional software to install and maintain on the computer because it was all part of the QuickBooks installation.

Intuit did a fantastic job of getting developers to move to the API integration method, positioning all those lovely 3rd party solutions for linkage via an Intuit.com account and, now, to QuickBooks Online.  Intuit is clearly favoring the QuickBooks Online edition and the API integration method available with that platform, and is telling developers that they must convert their customers to QBO in order to retain the easy connective ability they had with the desktop editions via Sync Manager.

Now that Intuit has announced the discontinuation of the REST API and the Sync Manager, what options do QuickBooks integration developers have, and how can customers using 3rd party integrations keep using them?  Options do remain, and they aren’t all that bad.  In fact, the options which remain continue to be the methods of choice for certain developers. These developers recognized early on that Intuit’s somewhat “lightweight” methods couldn’t handle the complexity or full functionality of their integrations facilitated their solutions using the SDK and never looked back (and still don’t).  For this community of developers – many of whom likely never considered trying to market their solutions in the Intuit app marketplace – the elimination of the REST API and Sync Manager don’t really matter.  They didn’t bother with them in the first place, just as they aren’t bothering with QBO.  Those solutions don’t fit their customers, anyway.

The QuickBooks desktop SDK (Software Development Kit) has been around for years, and using the SDK developers have been able to craft tight integrations between their solutions and the QuickBooks desktop products.  From payment plug-ins to fully integrated sales, customer relationship, inventory and manufacturing solutions – a broad range of integrated applications built with the SDK have been successfully deployed to QuickBooks customers all over the world.   Many applications which integrate with QuickBooks desktop solutions are desktop products themselves and are designed to work within the same desktop and network environment as QuickBooks, so there is no need to worry about “transport” of the data over the Internet.

For other solutions, such as online applications and services, there may be a need to exchange data via the Web. The QuickBooks Web Connector has also been a very popular solution for developers of applications that integrate data with QuickBooks.  The Web Connector is just what its name implies: it is a way to connect QuickBooks to the web and vice versa. With the Web Connector application and a web connector configuration file, developers could provide a method of exchanging data between QuickBooks desktop and another solution fairly simply.  While the Web Connector is quite useful in providing a means to transport integrated data to/from the QuickBooks desktop to an external system (like an online application), it only allows access to whatever data Intuit decides.  For this reason, many developers use both an SDK application and the Web Connector so their applications can access all data required and also have a web service available to transport it.

There are numerous implications relating to the sunset of QuickBooks REST API and Sync Manager, and another among them is the impact in hosted environments.  For customers who are (or might) benefit from hosted QuickBooks delivery models, what does the end-of-life of the Sync Manager mean?  Since the Sync Manager was basically built into QuickBooks desktop editions, it meant that there wasn’t any extra software to install or manage when a company wanted to adopt a Sync Manager-based 3rd party integrated solution. In a hosting environment, this means that the customer could easily add integrated applications to work with their hosted QuickBooks and the service provider might never even know it was being done.  There would be no additional software to install on the host servers; so many providers would simply be unaware that their customers were using these other solutions.

As developers return to SDK and Web Connector implementations in order to integrate with QuickBooks desktop, customers will ask their hosting providers to install the QWC (QuickBooks Web Connector) and/or integration software in their service.  In shared service delivery models, this may be virtually impossible to do without potential compromise to existing customers using those servers or other applications resident on the systems.  Hosting customers will not always understand that a “simple plug-in” actually represents installable software that must be secured, maintained, managed, and kept from improperly interacting with other software in the environment.  Some providers may not even be willing to work with the new integration software, while others may allow it but will not take adequate precautions to ensure proper and secure function.

Intuit has said to many constituent groups that its focus on desktop editions of QuickBooks will continue, and new certifications and benefits for desktop ProAdvisors (and continued development of interoperability with other solutions, like the Revel POS integration for QuickBooks desktop) give support to those statements.  Yet developers who support integrations with QuickBooks desktop are once again adjusting to the not infrequent changes Intuit makes to developer programs and philosophies.  The push to QBO and connected apps may be the focus for QuickBooks marketing dollars, but there are still quite a number of (very busy!) developers supplying solutions to businesses who don’t shop inside their QuickBooks software.

Joanie Mann Bunny FeetMake Sense?

J

[1] https://developer.intuit.com/blog/2014/09/08/timeline-to-discontinue-the-quickbooks-desktop-rest-api

[2] https://en.wikipedia.org/wiki/Representational_state_transfer

Paperless_468x80

Mobilizing QuickBooks Desktops

 Hosted QuickBooks for Remote and Mobile Access

There was a time not too long ago when the “thought leaders” in information technology said that the desktop is dead, and all software will be accessed via the web. (Note: I put “thought leaders” in quotes because industry thought leaders are often those with the greatest media influence.  After all, you can’t lead them if you can’t reach them, right?). The whole no software thing is a dramatic oversimplification of what is happening with computer software, but one thing is kind of coming true: nobody wants to be tied to their desktop.  It’s not that the desktop is dead… it’s just not all there is. For users of the desktop editions of Intuit QuickBooks software, the question really isn’t whether they intend to give up their familiar and trusted software to use a different, online solution. The question is how to use the QuickBooks desktop software they want in the cloud so they can use it on desktops that aren’t the primary desktop computer, or on mobile devices.

Computing technology has finally reached a level of accessibility that was previously only imagined in science fiction stories.  Communicating instantaneously with anyone anywhere around the world; accessing extensive (limitless?) libraries of information, art and music with a simple handheld device – these are the things that people do every day without a second thought.  Business users may even be able to access their business documents, email, contacts and appointments etc. from mobile devices, enabling a productive and functional mobile workforce.
desktop-appsYet the desktop remains as the primary workhorse for most business users. This is where the productivity applications live, where large spreadsheets and full-screen applications are run, and where keyboarders and production data entry users operate.  Tablets, touchscreens and mobile devices just don’t provide the same capabilities unless you tether them to full size monitors and keyboards.  Even then they may not because they might not run the same OS as the desktop.  The point is that the desktop hasn’t gone away and isn’t likely to any time soon.  Users may use more mobile apps and devices, but this isn’t diminishing use on the desktop as much as it augmenting it.  This is what fuels the interest in application hosting and virtual desktop computing models – the desire to mobilize desktop and network applications and working environments.

Hosting applications and data gives businesses the flexibility of working in desktop applications and accessing data just as if they were in the office, yet users may be located anywhere there is Internet connectivity. When the applications and the associated data are managed in the datacenter, businesses are able to centralize their information assets and manage them more effectively than if the data were distributed among multiple computers.  While most sync and share solutions require files to be downloaded to local computers in order to open and edit, a hosted application service with virtual desktops and file sharing provides a security model which keeps business data secure yet available for user access without compromising security by downloading information to the user device.

A hosted solution approach can make license utilization more efficient and compliance easier to maintain, too.  By enabling access to applications on a centralized platform and eliminating the installation and maintenance of software on individual computers, businesses reduce the reliance on local IT personnel to install and update applications and user accounts, and improve their ability to control application assignments and usage.

Hosting helps businesses take advantage of technology that would otherwise be unaffordable, and delivers the mobility and centralized management required to boost productivity and contain costs.  There is a high cost to managing a business network, and creating secure mobile access to that network can represent an exponential increase in IT spending (just to initially set up, not to mention ongoing costs for security management, monitoring and support). Rather than taking on the entire burden of service management and delivery directly, businesses electing to work with hosting providers find that they are able to focus more on business operation, strategy and growth – and spend less time worrying about the IT supporting them.  Costs are reduced, workers are empowered, and capabilities are increased while knowledge and process investments are preserved.  When it comes to mobilizing business applications like QuickBooks desktop editions, it all starts with a hosted approach.

Joanie Mann Bunny FeetMake Sense?

J