Ajay- What is PMML?

Mike- The Predictive Model Markup Language (PMML) is the leading standard for statistical and data mining models and supported by all leading analytics vendors and organizations. With PMML, it is straightforward to develop a model on one system using one application and deploy the model on another system using another application. PMML reduces complexity and bridges the gap between development and production deployment of predictive analytics.

PMML is governed by the Data Mining Group (DMG), an independent, vendor led consortium that develops data mining standards

Ajay- Why can PMML help any business?

Mike- PMML ensures business agility with respect to data mining, predictive analytics, and enterprise decision management. It provides one standard, one deployment process, across all applications, projects and business divisions. In this way, business stakeholders, analytic scientists, and IT are finally speaking the same language.

In the current global economic crisis more than ever, a company must become more efficient and optimize business processes to remain competitive. Predictive analytics is widely regarded as the next logical step, implementing more intelligent, real-time decisions across the enterprise.

However, the deployment of decisions based on predictive models and statistical algorithms has been a hurdle for many companies. Typically, it has been a complex, costly process to get such models integrated into operational systems. With the PMML standard, this no longer is the case. PMML simply eliminates the deployment complexity for predictive models.

A standard also provides choices among vendors, allowing us to implement best-of-breed solutions, and creating a common knowledge framework for internal teams – analytics, IT, and business – as well external vendors and consultants. In general, having a solid standard is a sign of a mature analytics industry, creating more options for users and, most importantly, propelling the total analytics market to the next level.

Ajay- Can PMML help your existing software in analytics and BI?

Mike- PMML has been widely accepted among vendors, almost all major analytics and business intelligence vendors already support the standard. If you have any such software package in-house, you most likely have PMML at your disposal already.

For example, you can develop your models in any of the tools that support PMML, e.g., SPSS, SAS, Microstrategy, or IBM, and then deploy that model in ADAPA, which is the Zementis decision engine. Or you can even choose from various open source tools, like R and KNIME.

Ajay- How does Zementis and ADAPA and PMML fit?

Mike- Zementis has been a avid supporter of the PMML standard and is very active in the development of the standard. We contributed to the PMML package for the open source R Project. Furthermore, we created a free PMML Converter tool which helps users to validate and correct PMML files from various vendors and convert legacy PMML files to the latest version of the standard.

Most prominently with ADAPA, Zementis launched the first cloud-computing scoring engine on the Amazon EC2 cloud. ADAPA is a highly scalable deployment, integration and execution platform for PMML-based predictive models. Not only does it give you all the benefits of being fully standards-based, using PMML and web services, but it also leverages the cloud for scalability and cost-effectiveness.

By being a Software as a Service (SaaS) application on Amazon EC2, ADAPA provides extreme flexibility, from casual usage which only costs a few dollars a month all the way to high-volume mission critical enterprise decision management which users can seamlessly launch in the United States or in European data centers.

Ajay- What are some examples where PMML helped companies save money?

Mike- For any consulting company focused on developing predictive analytics models for clients, PMML provides tremendous benefits, both for clients and service provider. In standardizing on PMML, it defines a clear deliverable – a PMML model – which clients can deploy instantly. No fixed requirements on which specific tools to choose for development or deployment, it is only important that the model adheres to the PMML standard which becomes the common interface between the business partners. This eliminates miscommunication and lowers the overall project cost. Another example is where a company has taken advantage of the capability to move models instantly from development to operational deployment. It allows them to quickly update models based on market conditions, say in the area of risk management and fraud detection, or to roll out new marketing campaigns.

Personally, I think the biggest opportunities are still ahead of us as more and more businesses embrace operational predictive analytics. The true value of PMML is to facilitate a real-time decision environment where we leverage predictive models in every business process, at every customer touch point and on-demand to maximize value

Ajay- Where can I find more information about PMML?

Mike- First there is the Data Mining Group (DMG) web site at http://www.dmg.org

I strongly encourage any company that has a significant interest in predictive analytics to become a member and help drive the development of the standard.

We also created a knowledge base of PMML-related information at http://www.predictive-analytics.info and there is a PMML interest group on Linked

In http://www.linkedin.com/groupRegistration?gid=2328634

This group is more geared toward a general discussion forum for business benefits and end-user questions, and it is a great way to get started with PMML.

Last but not least, the Zementis web site at http://www.zementis.com

It contains various PMML example files, the PMML Converter tool, as well links to PMML resource pages on the web.

For more on Michael Zeller and Zementis read his earlier interview at http://decisionstats.wordpress.com/2009/02/03/interview-michael-zeller-ceozementis-2/

Ajay- Describe your career journey across the full development cycle of Business Intelligence.

Ken- I started my career in the early 80’s in the airline industry, where I worked as an application programmer and later as a systems programmer. I took a computer science degree by night. The airline industry was one of the first to implement computer systems in the ‘60s, and the legacy of being an early adaptor was that airline reservation systems were developed in Assembler. Remarkable as it sounds now, as application programmers, we wrote our own file access methods. Even more remarkable, as systems programmers, we modified the IBM supplied Operating System, originally known as the Airline Control Program (ACP), later renamed as Transaction Processing Facility (TPF). The late ‘80s saw the development of Global “Computer Reservations Systems” (CRS systems) including AMADEUS and GALILEO. I moved from Aer Lingus, a small Irish airline, to work in London on the British Airways systems, to enable the British Airways systems share information and communicate with the new Global CRS systems.

I learnt very important lessons during those years.

* The criticality of standards

* The drive for interoperability of systems

* The drive towards information sharing

* The drive away from bespoke development

In the 90’s I returned to Dublin, where I worked as an independent consultant with IBM on many data intensive projects. On one project I was lead developer in the IBM Dublin Laboratory on the development of the Data Replication tool called “Data Propagator NonRelational”. This tool automatically propagates updates made on IMS databases to DB2 databases. On this project, we successfully piloted using the Cleanroom Development Method, as part of IBM’s derive towards Six Sigma quality.

In the past 15 years I have moved away from IT towards the business. I describe myself as a Hybrid. I believe there is a serious communications gap between business users and IT, and this is a frequent cause of project failures. I seek to bridge that gap. I ensure that requirements are clear, measurable, testable, and capable of being easily understood and signed off by business owners.

One of my favorite programmes was Euro Changeover, This was a hugely data intensive programme. It was the largest changeover undertaken by European Financial Institutions. I worked as an independent consultant with the IBM Euro Centre of Competence. I developed changeover strategies for a number of Irish Enterprises, and was the End to End IT changeover process owner in a major Irish bank. Every application and every data store holding currency sensitive data (not just amounts, but currency signs etc.) had to be converted at exactly the same time to ensure that all systems successfully switched to euro processing on 1st January 2002.

I learnt many, many lasting lessons about data the hard way on Euro Changeover programmes, such as:

* The extent to which seemingly separate applications share operational data – often without the knowledge of the owning application.

* The extent to which business users use (abuse) data fields to hold information never intended for the data field.

* The critical distinction between the underlying data (in a data store) and the information displayed to a business user.

I have worked primarily on what I call “End of food chain” projects and programmes, such as Single View of Customer, data migrations, and data population of repositories for BASEL II and Anti Money Laundering (AML) systems. Business Intelligence is another example of an “End of food chain” project. “End of food-chain” projects share the following characteristics:

* Dependent on existing data

* No control over the quality of existing data they depend on

* No control over the data entry processes by which the data they require is captured.

* The data required may have been captured many years previously.

Recently, I have shared my experience of “Enterprise wide data issues” in a series of posts on my blog, together with a process for assessing the status of those issues within an Enterprise (more details). In my experience, the success of a Business Intelligence programme and the ease with which an Enterprise completes “End of food chain” data dependent programmes directly depends on the status of the common Enterprise Wide data issues I have identified.

Ajay -Describe the educational scene for science graduates in Ireland. What steps do you think governments and universities can do to better teach science and keep young people excited about it?

Ken- I am not in a position to comment on the educational scene for science graduates in Ireland. However, I can say that currently there are insufficient numbers of school children studying science in primary and 2nd level education. There is a need to excite young people about science. There is a need for more interactive science museums, like W5 in Belfast which is hugely successful. Kids love to get involved, and practical science can be great fun.

Ajay- What are some of the key trends in business intelligence that you have seen-

Ken- Since the earliest days of my career, I have seen an ever increasing move towards standards based interoperability of systems, and interchange of data. This has accelerated dramatically in recent years. This is the good news. Further good news is the drive towards the use of external reference databases to verify the accuracy of data, at point of data entry (See blog post on Upstream prevention by Henrik Liliendahl Sørensen). One example of this drive is cloud based verification services from new companies like Ireland based Clavis Technology.

The harsh reality is that “Old hardware goes into museums, while old software goes into production every night”. Enterprises have invested vast amounts of money in legacy applications over decades. These legacy systems access legacy data in legacy data stores. This legacy data will continue to pose challenges in the delivery of Business Intelligence to the Business community that needs it. These challenges will continue to provide opportunities for Data Quality professionals.

Ajay- What is going to be the next fundamental change in this industry in your opinion?

Ken- The financial crisis will result in increased regulatory requirements. This will be good news for the Business Intelligence / Data Quality industry. In time, it will no longer be sufficient to provide the regulator with ‘just’ the information requested. The regulator will want to see the process by which the information was gathered; the process controls, and evidence of the quality the underlying data from which the information was derived. This move will result in funding for Data Governance programmes, which will lead to increased innovation in our industry.

Ajay- Describe your startup Map My Business, your target customer and your vision for it.

Ken- I started MapMyBusiness.com as a “recession buster”. Ireland was hit particularly hard by the financial crisis. I had become over dependent on the financial services industry, and a blanket ban on the use of external consultants left me with no option but to reinvent myself. MapMyBusiness.com helps small businesses to attract clients, by getting them on Google page one. Having been burnt by an over dependence on one industry, my vision is to diversify. I believe that Data Governance is industry independent, and I am focussing on increasing my customer base for my Data Governance consultancy skills, via my company Professional IT Personnel Ltd.

Ajay- What do you do when not working with customers or blogging on your website?

Ken- I try to achieve a reasonable work/life balance. I am married with two children aged 12 and 10, and like to spend time with them, especially outdoors, walking, hiking, playing tennis etc. I am involved in my community, lobbying for improved cycling infrastructure in our area (more details). Ireland, like most countries, is facing an obesity epidemic, due to an increasingly sedentary lifestyle. Too many people get little or no exercise, and don’t have the time, willpower, or perhaps money, to regularly work out in a gym. By including “Active Travel” in our daily lives – by walking or cycling to schools and local amenities, we can get enough physical exercise to prevent obesity, and obesity related health problems. We need to make our cities, towns and villages more pedestrian and cyclist friendly, to encourage “active travel”. My voluntary work in this area introduced me to mapping (see example), and enabled me to set up MapMyBusiness.com.

Biography-

Ken O’Connor is an independent IT Consultant with almost 30 years of work experience. He specialises in Data: Data Migration, Data Population, Data Governance, Data Quality, Data Profiling…His  company is called Professional IT Personnel Ltd.

Ken started his blog (Ken O’ Connor Data Consultant) to share his experience and to learn from the experience of others.   Dylan Jones, editor of dataqualitypro, describe Ken as a “grizzled veteran”, with almost 30 years experience across the full development lifecycle.

Shawn: My career journey has spanned over a decade in several Silicon Valley technology companies.  In both high school and my college studies at Princeton, I had a fervent interest in math and quantitative economics.  Silicon Valley drew me to companies like upstart procurement software maker Ariba and database giant Oracle.  I continued my studies by returning to get a Master’s in Management Science at Stanford before going on to lead core storage systems for nearly 5 years at NetApp and subsequently Aster.

Science (whether it is math, physics, economics, or the hard engineering sciences) provides a solid foundation.  It teaches you to think and test your assumptions – those are valuable skills that can lead to a both a financially lucrative and personally inspiring career.

Shawn: Let’s start with the database guys – Oracle and Teradata.  They focus on structured data – data that has a logical schema and is manipulated via a standards-based structured query language (SQL).  Oracle tries to be everything to everyone – it does OLTP (low-latency transactions like credit card or stock trade execution apps) and some data warehousing (typically summary reporting).  Oracle’s data warehouse is not known for large-scale data warehousing and is more often used for back-office reporting.

Teradata is focused on data warehousing and scales very well, but is extremely expensive – it runs on high-end custom hardware and takes a mainframe approach to data processing.  This approach makes less sense as commodity hardware becomes more compute-rich and better software comes along to support large-scale MPP data warehousing.

SAS is very different – it’s not a relational database. It really offers an application platform for data analysis, specifically data mining.  Unlike Oracle and Teradata which is used by SQL developers and managed by DBAs, SAS is typically run in business units by data analysts – for example a quantitative marketing analyst, a statistician/mathematician, or a savvy engineer with a data mining/math background.  SAS is used to try to find patterns, understand behaviors, and offer predictive analytics that enable businesses to identify trends and make smarter decisions than their competitors.

Hadoop offers an open-source framework for large-scale data processing.  MapReduce is a component of Hadoop, which also contains multiple other modules including a distributed filesystem (HDFS).  MapReduce offers a programming paradigm for distributed computing (a parallel data flow processing framework).

Both Hadoop and MapReduce are catered toward the application developer or programmer.  It’s not catered for enterprise data centers or IT.  If you have a finite project in a line of business and want to get it done, Hadoop offers a low-cost way to do this.  For example, if you want to do large-scale data munging like aggregations, transformations, manipulations of unstructured data – Hadoop offers a solution for this without compromising on the performance of your main data warehouse.  Once the data munging is finished, the post-processed data set can be loaded into a database for interactive analysis or analytics. It is a great combination of big data technologies for certain use-cases.

Aster takes a very unique approach.  Our Aster nCluster software offers the best of all worlds – we offer the potential for deep analytics of SAS, the low-cost scalability and parallel processing of Hadoop/MapReduce, and the structured data advantages (schema, SQL, ACID compliance and transactional integrity, indexes, etc) of a relational database like Teradata and Oracle.  Often, we find complementary approaches and therefore view SAS and Hadoop/MapReduce as synergistic to a complete solution.  Data warehouses like Teradata and Oracle tend to be more competitive.

Shawn: Aster was the first-to-market to offer In-Database MapReduce, which provides the standards and familiarity of SQL and databases with the analytic power of MapReduce.  This is very unique as it offers technical developers and application programmers to write embedded procedural algorithms once, upload it, and allow business analysts or IT folks (SQL developers, DBAs, etc) to invoke these SQL-MapReduce functions forever.

It is highly polymorphic (re-usable), highly fault-tolerant, highly flexible (any language – Java, Python, Ruby, Perl, R statistical language, C# in the .NET world, etc) and natively massively parallel – all of which differentiate these SQL extensions from traditional dumb user-defined functions (UDFs).

Shawn: Aster has dozens of production customers including big-names like MySpace, LinkedIn, Akamai, Full Tilt Poker, comScore, and several yet-to-be-named retail and financial service accounts.  We have quantified proof points that show orders of magnitude improvements in scalability, performance, and analytic insights compared to incumbent or competitor solutions.  Our highly referenceable customers would be happy to discuss their positive experiences with the CEO.

But taking a step back, there’s a fundamental concept that this CEO needs to first understand.  The world is changing – data growth is proliferating due to the digitization of so many applications and the emergence of unstructured data and new data types.  Like the book “Competing on Analytics”, the world is shifting to a paradigm where companies that don’t take risks and push the limits on analytics will die like the dinosaurs.

IDC is projecting 10x+ growth in data over the next few years to zetabytes of aggregate data driven by digitization (Internet, digital television, RFID, etc).  The data is there and in order to compete effectively and understand your customers more intimately, you need a large-scale analytics solution like the one Aster nCluster offers.  If you hold off on experimentation and innovation, it will be too late by the time you realize you have a problem at hand.

Shawn: Very important.  I hang out with my wife most weekends – we do a lot of outdoors activities like hiking and gardening.  In Silicon Valley, it’s all too easy to get caught up in the rush of things.  Taking breaks, especially during the weekend, is important to recharge and re-energize to be as productive as possible.

Shawn: We’re always looking for smart, innovative, and entrepreneurial new college grads and interns, especially on the technical side.  So if you are a computer science major or recent grad or graduate student, feel free to contact us for opportunities.

What makes Aster exciting is 2 things –

first, the people.  Everyone is very smart and innovative so you learn a tremendous amount, which is personally gratifying and professionally useful long-term.

Second, Aster is changing the world!

Distributed systems computing focused on big data processing and analytics – these are massive game-changers that will fundamentally change the landscape in data warehousing and analytics.  Traditional databases have been a oligopoly for over a generation – they haven’t been challenged and so the 1970’s based technology has stuck around.  The emergence of big data and low-cost commodity hardware has created a unique opportunity to carve out a brand new market…

what gets me pumped every day is I have the ability to contribute to a pioneer that is quickly becoming Silicon Valley’s next great success story!

Biography-

Over the past decade, Shawn has led product management for some of Silicon Valley’s most successful and innovative technology companies.  Most recently, he spent nearly 5 years at Network Appliance leading Core Systems storage product management, where he oversaw the development of high availability software and Storage Systems hardware products that grew in annual revenue from $200M to nearly $800M.  Prior to NetApp, Shawn held senior product management and corporate strategy roles at Oracle Corporation and Ariba Inc.

Shawn holds an M.S. in Management Science and engineering from Stanford University, where he was awarded the Valentine Fellowship (endowed by Don Valentine of Sequoia Capital).  He also received a B.A. with high honors from Princeton University.

About Aster

Aster Data Systems is a proven leader in high-performance database systems for data warehousing and analytics – the first DBMS to tightly integrate SQL with MapReduce – providing deep insights on data analyzed on clusters of low-cost commodity hardware. The AsternCluster database cost-effectively powers frontline analytic applications for companies such as MySpace, aCerno (an Akamai company), and ShareThis.

Running on low-cost off-the-shelf hardware, and providing ‘hands-free’ administration, Aster enables enterprises to meet their data warehousing needs within their budget. Aster is headquartered in San Carlos, California and is backed by Sequoia Capital, JAFCO Ventures, IVP, Cambrian Ventures, and First-Round Capital, as well as industry visionaries including David Cheriton and Ron Conway.

Ajay- Describe your career as a science student to an author of science and strategy books.
Redman: I took the usual biology, chemistry, and physics classes in college.  And I worked closely with oceanographers in graduate school.  More importantly, I learned directly from two masters.  First, was Dr. Basu, who was at Florida State when I was.  He thought more deeply and clearly about the nature of data and what we can learn from them than anyone I’ve met since.  And second is people in the Bell Labs’ community who were passionate about making communications better. What I learned there was you don’t always need “scientific proof” to mover forward.

Ajay- What kind of bailout do you think the Government can give to the importance of science education in this country.

Redman: I don’t think the government should bail science education per se. Science departments should compete for students just like the English and anthropology departments do.  At the same time, I do think the government should support some audacious goals, such as slowing global warming or energy independence.  These could well have the effect of increasing demand for scientists and science education.

Ajay- Describe your motivations for writing your book Data Driven-Profiting from your most important business asset.

Redman: Frankly I was frustrated.  I’ve spent the last twenty years on data quality and organizations that improve gain enormous benefit.  But so few do.  I set out to figure out why that was and what to do about it.

Ajay- What can various segments of readers learn from this book-
a college student, a manager, a CTO, a financial investor and a business intelligence vendor.

Redman: I narrowed my focus to the business leader and I want him or her to take away three points.  First, data should be managed as aggressively and professionally as your other assets.  Second, they are unlike other assets in some really important ways and you’ll have to learn how to manage them.  Third, improving quality is a great place to start.

Ajay- Garbage in Garbage out- How much money and time do you believe is given to data quality in data projects.

Redman:   By this I assume you mean data warehouse, BI, and other tech projects.  And the answer is “not near enough.”  And it shows in the low success rate of those projects.

Ajay-Consider a hypothetical scenario- Instead of creating and selling fancy algorithms , a business intelligence vendor uses simple Pareto principle to focus on data quality and design during data projects. How successful do you think that would be?

Redman: I can’t speak to the market, but I do know that if organizations are loaded with problems and opportunities.  They could make great progress on most important ones if could clearly state the problem and bring high-quality data and simple techniques to bear.  But there are a few that require high-powered algorithms.  Unfortunately those require high-quality data as well.

Ajay- How and when did you first earn the nickname “Data Doc”. Who gave it to you and would you rather be known by some other names.

Redman: One of my clients started calling me that about a dozen years ago.  But I felt uncomfortable and didn’t put it on my business card until about five years ago.  I’ve grown to really like it.

Ajay- The pioneering work at AT & T Bell laboratories and at Palo Alto laboratory- who do you think are the 21st century successors of these laboratories. Do you think lab work has become too commercialized even in respected laboratories like Microsoft Research and Google’s research in mathematics.

Redman: I don’t know.  It may be that the circumstances of the 20^th century were conducive to such labs and they’ll never happen again.  You have to remember two things about Bell Labs.  First, was the cross-fertilization that stemmed from having leading-edge work in dozens of areas.  Second, the goal is not just invention, but innovation, the end-to-end process which starts with invention and ends with products in the market.  AT&T, Bell Labs’ parent, was quite good at turning invention to product.  These points lead me to think that the commercial aspect of laboratory work is so much the better.

Ajay-What does ” The Data Doc” do to relax and maintain a work life balance. How important do you think is work-life balance for creative people and researchers.

Redman: I think everyone needs a balance, not just creative people.  Two things have made this easier for me.  First, I like what I do.  A lot of days it is hard to distinguish “work” from “play.”  Second is my bride of thirty-three years, Nancy.  She doesn’t let me go overboard too often.

Biography-

Dr. Thomas C. Redman is President of Navesink Consulting Group, based in Little Silver, NJ.  Known by many as “the Data Doc” (though “Tom” works too), Dr. Redman was the first to extend quality principles to data and information.  By advancing the body of knowledge, his innovations have raised the standard of data quality in today’s information-based economy.

Dr. Redman conceived the Data Quality Lab at AT&T Bell Laboratories in 1987 and led it until 1995.  There he and his team developed the first methods for improving data quality and applied them to important business problems, saving AT&T tens of millions of dollars. He started Navesink Consulting Group in 1996 to help other organizations improve their data, while simultaneously lowering operating costs, increasing revenues, and improving customer satisfaction and business relationships.

Since then – armed with proven, repeatable tools, techniques and practical advice – Dr. Redman has helped clients in fields ranging from telecommunications, financial services, and dot coms, to logistics, consumer goods, and government agencies. His work has helped organizations understand the importance of high-quality data, start their data quality programs, and also save millions of dollars per year.

Dr. Redman holds a Ph.D. in statistics from Florida State University.  He is an internationally renowned lecturer and the author of numerous papers, including “Data Quality for Competitive Advantage” (Sloan Management Review, Winter 1995) and “Data as a Resource: Properties, Implications, and Prescriptions” (Sloan Management Review, Fall 1998). He has written four books: Data Driven (Harvard Business School Press, 2008), Data Quality: The Field Guide (Butterworth-Heinemann, 2001), Data Quality for the Information Age (Artech, 1996) and Data Quality: Management and Technology (Bantam, 1992). He was also invited to contribute two chapters to Juran’s Quality Handbook, Fifth Edition (McGraw Hill, 1999). Dr. Redman holds two patents.

About Navesink Consulting Group (http://www.dataqualitysolutions.com/ )

Navesink Consulting Group was formed in 1996 and was the first company to focus on data quality.  Led by Dr. Thomas Redman, “the Data Doc” and former AT&T Bell Labs director, we have helped clients understand the importance of high-quality data, start their data quality programs, and save millions of dollars per year.

Our approach is not a cobbling together of ill-fitting ideas and assertions – it is based on rigorous scientific principles that have been field-tested in many industries, including financial services (see more under “Our clients”).  We offer no silver bullets; we don’t even offer shortcuts. Improving data quality is hard work.

But with a dedicated effort, you should expect order-of-magnitude improvements and, as a direct result, an enormous boost in your ability to manage risk, steer a course through the crisis, and get back on the growth curve.

Ultimately, Navesink Consulting brings tangible, sustainable improvement in your business performance as a result of superior quality data.

Ajay- Describe your career as a BI consultant.

Straycat- I’m an aerospace engineer who had to turn to IT right after graduation because of the Italian aerospace industry crisis in the first half of the 90’s . My first job was by the company now called Accenture, as a simple developer. I was part of a large project for a large US food corporation.

We built an enterprise level reporting and budgeting system based on what was later to become Hyperion. After that I had various experiences, always as an IT professional, always focusing on BI or related subjects. I worked for the Milan Airport Authority, the l’Oreal group and couple of local software houses. Now I’m a project manager by a large Italian consulting firm but, most of all, I’m a bootstrapping entrepreneur.

Ajay- How do you think we can teach BI at an early stage to young students.

Straycat- I think that the main problem resides in the naïve university approach toward business data analysis. Collecting data is considered trivial compared to other related subjects.

Data availability is often given for granted, then equations are written upon them. No use to say it is not trivial at all and there is an entire class of problems which students are not aware of.

A few lessons spent focusing on data quality, aggregations, measure definitions etc are enough to create the necessary awareness of the problem. It’s no longer cool telling to be ignorant on the subject!

Ajay- Describe the most challenging project you ever did. Name a project which led to the biggest dollar impact.

Straycat- About three years ago we signed a contract with a large fashion firm here in Italy to reengineer their entire business intelligence setup. It has been a project ranging from sales to production, from accounting to human resources.

It impacted almost a thousand users in six different time zones. The main challenge we had to tackle was the fragmentation of their legacy BI systems, which produced different jargon and practices across the corporation. We changed the database and the presentation layer, built a modern datawarehouse, and worked relentlessly on change management.

I can’t disclose figures but the new unified system shed light on some bad practices, revealed inefficiencies and provided a whole new set of analytics that increased market awareness.

Ajay- Describe your start-up StraySoft and what it is hoping to accomplish.

Straycat-

StraySoft is a small and fresh startup devoted to build Business Intelligence applications.

It produces Viney@rd an Excel/SQL Server based spreadsheet automation and BI tool.

I have personal reasons for embarking in such a project but the kick off came from a sudden realization. Despite the terrific sophistication level provided by current BI tools, the one thing each and every user wants is to have data in MS Excel.

This is simply a fact, users get data, elaborate them and make Excel reports. It’s not a matter of features, people feel in full control only when they have an Excel file.

Why? Because Excel is able to address a single cell, and the figures within can be adjusted at will and saved in a familiar place like C: .

So, the original idea (2 years and one half ago) was to create a tool to refresh a complex layout without disrupting it i.e. a tool which could address query results into single cells.

This can be done by Excel alone but it’s far too difficult even for and advanced user.

Viney@rd features this but I soon realized that, if I wanted to go down this path, I had to tackle a second issue: the data provided by the systems are never the data required by the user. I’m not talking about bad analysis or wrong KPIs; even if the architects did everything fine, the human brain works according to categories that often are not saved within a database.

Example: you are a salesman and you have 4 customers who make the 75% of your business. Plus you have 40 customers who make the other 25% of your business.

Question: “how many customers do you have?”, reply: “I have 4, customer A,B.C and D. A is bla bla bla, B is bla bla bla, C etc. etc. Oh, by the way I have some others but they are marginal.”

The salesman needs any kind of information about the 4, and just few hints about the rest; every detailed information about the rest is perceived as clutter. He needs a screen with 4 ultra detailed sheets per customer, not a customer ABC report with 44 rows.

So far nothing revolutionary, what is revolutionary is that the user himself must be able to tag the 4 main customers according to his own perception of the customer importance.

If one of the small customers is going to place a large order, than it must become important as well and should immediately take the fifth place, to be automatically demoted when the opportunity expires.

The point is that these rules are defined heuristically by the human brain and have so many exceptions that can be handled only by a human brain. This consideration led to implementing the unique feature of letting the users change their data directly by an Excel table.

The Viney@rd database is easy to be fed by traditional techniques but Excel sheet data can be saved within as well. This gives the best of both worlds, a central repository for “conventional” data, so no more “spreadsheet hell” nightmares, but the ability to classify and adjust the data still working in Excel.

This approach has limits, specifically when we talk about large amounts of data, I’m the first to admit it, but I still think that it’s the one thing that can popularize BI among business users.

When large vendors will embrace this, I’ll remind them this interview! ) Viney@rd now is in its infancy but already implements these two core features.

There are a lot of things to do, and many features to add to take it to a full corporate level, but I enjoy the process so much that I can’t stop working on it!

I’ve been asked “What if I buy from you and you go belly up next year?”. My reply is that you must shoot me to stop me from working on it! I still have a long list of features to implement and I’m not going to dismiss the fun!

For example, did you ever notice that people think naturally in terms of information streams ….?

I’ll consider myself successful when 3 conditions will be met:

a) I’ll have a body of satisfied users which had their working lives improved by my products

b) I’ll make a living out of StraySoft together with the employees, when I’ll have some

c) people will think to me as the Business Intelligence “enfant terrible”.

Ajay-  What do you do in your spare time ?

StrayCat- Sorry? What’s spare time? Jokes apart, I devote time to my wife, who’s really supportive in this effort. Late at night, before falling asleep, I’m used to read for half an hour: I’m passionate about history; but the events I really never miss are the Italian National Rugby Union Team matches.

Ajay- why do you tweet using the name Stray Cat ?

Augusto- I named the company StraySoft after the adoption of a stray cat; the full story is told here http://www.straysoft.com/dblog/articolo.asp?id=30.

The twitter name came as natural as naming the company. I know that someone may find it awkward but I feel like going upstream on that! Secondly, I want to keep my consulting activity and StraySoft totally separate for a matter of convenience. I did not, and will never propose my product to my consulting customers.

Ajay-  What visible trends in Business Intelligence do you fore see for the next two to three years.

Augusto- The #1 trend is that all the main vendors (excluding Microsoft, which already did) finally realized that there’s a midrange market which needs BI more than ever.

What they’re doing wrong is targeting this segment with the same enterprise class tools which miss the few key features required by this market.

The #2 trend is the rising of workgroup BI and the new dignity given to informal analysis. This is a whole new approach I do not share completely but I admit it has its strengths.

The #3 trend is at the opposite side of the spectrum; unconventional databases (columnar stores, appliances etc.) are becoming increasingly popular to manage very large amounts of data.

There are two fake trends: Clouds and SaaS. They’ll get a share of the market but will not become, in the foreseeable future, the reference architecture. Thank you again for giving me voice. All the best. Augusto Albeghi

Ajay-To know more on Augusto’s startup and Vineyard please see www.straysoft.com

Businesses realize that there’s more to information delivery than just distributing reports; companies rely on data analysis to support operational decision making.- Evan Levy

Ajay- Describe your career in science and technology.

Evan Levy- My formal “science and technology career” started during college. I received degrees in Electrical Engineering and Computer Science from Duke University. While in school, I had several programming jobs both on and off campus: a systems integration team within a large government contractor; a research study within the Psychology Department; and the university Computation Center. After graduation, I worked at a database computer startup (Teradata) for several years beforeco-founding Baseline Consulting. I’ve been here 18 years.

Ajay- How would help solve the problem of chronic technology worker shortage in the United States?.

Evan- I think the “technology worker shortage” in the US is a problem with multiple facets. I don’t think it can be addressed simply by “throwing bodies” at the problem. I tend to view IT as two distinct areas: processing operations and information delivery. Processing operations includes the development and maintenance of all of the operational computing platforms (applications, mainframes, servers, desktop systems, processing infrastructure, etc.) Information delivery focuses on a company’s data and the associated integration and analytical infrastructure (data integration, analysis tools, processing platforms, etc.)

There’s been a fundamental shift in operational systems development over the past 15 years. The belief that most companies had unique, specialized business processes requiring custom developed applications proved unrealistic. The costs, resources, and long timelines associated with these activities weren’t practical in today’s business environment. Most companies have been willing to shift to “packaged applications” that allowed them to evaluate and trade business process customization for time. This has caused skills within development teams to change dramatically; the need for business process and data analysis skills has exploded.

The growth and adoption of business analysis as a core business capability has also changed the approach to technology development. The time of distributing standard reports across the company in the weekly or monthly format simply isn’t sufficient any more. Businesses realize that there’s more to information delivery than just distributing reports; companies rely on data analysis to support operational decision making. Detailed data analysis and exception reporting isn’t a luxury, it’s become part of the core business functions required by most companies. This has caused business users to become much more sophisticated in data analysis – and it has caused a need for IT to expand their focus from simply providing applications to having to provide detailed data to business users.

I think these two changes have caused a significant shift in the “technology worker shortage”. I don’t think we’re living in the time where the application is the key business asset for companies. I think folk are beginning to realize that the key asset is data. The only way to address the backlog in applications is to fundamentally shift how we approach the problem. Our users are learning to solve their problems with desktop tools; we need to be able to deliver detailed data to them in a more efficient manner.

The only way to address this challenge is for the technology worker to become educated on their users’ business practices and methods. I don’t think throwing thousands of programmers at the issue will solve the current problem. I think we need to capitalize on the skills that already exist within our user communities and position IT resources to make them more self sufficient. I don’t think there’s any short cut to building and maintaining operational applications; programmers will always be needed for that. However, in the growth area of business and data analysis, I think we need to take an entirely different tact. We need to make users more self-sufficient with data.

Ajay- Unemployment in the United States is now touching 10 % yet millions of jobs that went overseas remain there. How good or bad has the technology sector been affected by offshoring compared to other sectors.

Evan- I’m no economist, so I’m afraid I can’t offer much of an opinion regarding the impact of offshoring technology jobs. I do know that in business, there will always be a desire to build and deliver products in as cost-efficient a manner as possible. We’ve seen numerous industries expand through offshoring and outsourcing. It shouldn’t be a surprise to anyone that the technology industry is maturing in much the same manner as every other industry. We all expect companies to stay competitive through managing costs; offshoring and outsourcing is an accepted practice.

I think the impact to offshoring and outsourcing will continue to impact the technology sector. We continue to see companies reevaluate their technology investments. As some technologies mature, evolve, and become commodities, I suspect we’ll continue to see jobs associated with those technologies to be outsourcd.

I think the challenge to those of us that work within the technology industry is to continually invest and grow our skills. As any industry matures, the products transition from being specialized to commodities. Look at the internet and web applications and tools. Prior to 2000, building even a simple website proved expensive and resource intensive. Today, most anyone with fundamental pc skills can build their own website. This industry has collapsed in size – but many continue in that space because they’ve grown and expanded their skills. A look at the sophistication of today’s websites reflects this shift.

Ajay- What data solutions would you recommend for the United States government to better channel its stimulus spending.

Evan- I actually think the government has taken an interesting approach with some aspects of the current stimulus spending. I can’t remember when it was possible for any of us to quickly determine where and how federal money was distributed. Today, there are several websites that provide detailed information identifying individual projects and their related funding levels. I wish this type of detailed data was made available for federal spending related to Hurricane Katrina, or the activities in Iraq or Afghanistan. It would provide clarity to where our tax dollars go and raise visibility to the inappropriate distribution of funds.

It think it would be valuable that any and all government spending to be made available to the public in a simple online manner.

Ajay- Do you think Business Intelligence is a male dominated sector. If so, why?

Evan- I’m not sure BI is any more male-dominated than any other IT area. But I’ll say one thing: we need more women in IT. It’s not about gender-specific skills or even about unique talents. It’s just about balance and perspective. Some of my best friends are women! Seriously, I think women do bring some cultural and knowledge assets to the table that just make the overall environment better for everyone. The women who work for me are so exceptional that I should probably be working for them.

When it comes to BI, Cindi Howson—a BI thought leader in her own right and someone who knows a LOT about the vendor space—wrote a great blog post about women in BI. It mentions my partner Jill, whom you interviewed a few months ago. Jill and Cindi are only two of a stellar group of women in BI. (I’d call them “BI babes” but I’d be seriously hurt if I did that.) But I think Cindi’s blog says it all.

Ajay- What do you for relaxing? How important are hobbies and family life for busy career professionals.

Evan- I’m a strong believer in the balance of work and play. My colleagues and staff at Baseline work hard with our clients. Many of our client projects require time and travel flexibility that doesn’t align with the traditional 9—5 world. It’s important for individuals to spend time with their friends and families – to enjoy the things that are outside of their jobs. Personally I spend my time volunteering with the YMCA. It’s a life-long cause for me, and the source of many of my best friendships.

Ajay-  What are your views on mis-selling in consulting- selling something which you are not really an expert of. Does this happen in your opinion in BI.

Evan- That’s a pretty interesting question. I’m sure we all know about situations where an aggressive sales person made impractical promises to address business challenges and established unrealistic expectations. Individuals are driven by their company’s incentive system. I find that when a company rewards its team members on client satisfaction and project success (instead of simply the numbers), mis-selling rarely occurs. I often recommend that our clients ask their suppliers how their sales people and client teams are rewarded. We often see those questions in RFPs.

Most of the consulting problems we see aren’t related to aggressive selling, but simply a gap between the requirements and the solution. While this sounds trite – we often find that the solution providers don’t fully understand the problem they are solving. Whether it’s because the problem wasn’t well understood (by the solution provider) or well analyzed and described (by the prospect) is sometimes impossible to determine; it’s usually a combination of both.

Preventing (or limiting) these surprises is very doable; short-term and small deliverables, frequent and thorough project reviews, and measureable acceptance criteria is a good place to start.

I’ll be the first to admit that at Baseline, we’re much better at delivery than we are at sales. This means that we don’t chase deals, but when we get them we deliver. We like to delight our clients. And our consultants really know their stuff. Because of that we have great client references, for which we’re grateful.

Biography

Evan Levy is a partner and co-founder of Baseline Consulting. Evan has spent his career leading both practitioners and executives in delivering a range of IT solutions. In addition to his executive management responsibilities at Baseline, he regularly oversees high-profile systems integration projects for key clients such as Charles Schwab, Verizon, State of Michigan, and CheckFree.

Evan also advises software vendors in the areas of product planning, and continues to counsel the executive and investment communities in applying advanced technologies to key business initiatives. Evan has been known to shave off his beard on a bet. He can whistle most of the songs in the Sesame Street oeuvre, has a thing for silicone kitchen implements, and helped design the data warehouse at one of those superstores that is inevitably coming to a neighborhood near you.

Author

Evan writes frequently for leading industry publications, focusing on the financial payback of IT investments, architectural best practices, and data integration alternatives. He is a regular online contributor to DMReview.com and SearchDataManagement.com.

Evan is also co-author of the book, Customer Data Integration (John Wiley and Sons, 2006), which describes the business breakthroughs achieved with integrated customer data, and explains how to make CDI work. Evan also writes a regular blog for Baseline http://www.evanjlevy.com/

Industry Leader

Evan has been a thought leader at major industry and vendor conferences, including the American Marketing Association, DAMA International, MDM Summit, MDM Insight, and TechTarget conferences. He is a faculty member of TDWI and delivers regular presentations on data integration alternatives. Recent seminars have focused on the application of emerging technologies and use cases for master data management and data integration solutions.

Baseline Consulting, an acknowledged leader in information design and deployment, helps companies enhance the value of their enterprise data, improve business performance, and achieve self-sufficiency in managing data as a corporate asset. Baseline provides business consulting and technical implementation services in four practice areas: Data Warehousing, Data Integration, Business Analytics, and Data Governance. Founded in 1991 and headquartered in Los Angeles, California, Baseline changes how companies leverage information. To learn more, visit Baseline’s website at www.baseline-consulting.com.

James is hosting a webinar series on decision manaement, predictive analytics and business rules this fall. You can check out the webinars and register for some or all at https://decisionmanagement.omnovia.com/registration

Here is an interview with James Taylor, a leading consultant and evangelist in the emerging field of converging decision management.

Ajay- Describe your career in science. What fascinates you with reporting on this segment. How would you interest freshmen students in taking up statistics and math courses.

James- I took Geological Geophysics and Mathematics in college but graduated in a year when the oil price was in free fall and never worked in geophysics. Since then I have worked in computers, mostly focused on how they can be applied rather than on how they work. I am not sure I would say that this represents a career in science so much a career enabled by science and, increasingly, watching science.

As far as math goes I actually think the problem is at the other end of the spectrum. Far too many people leave school without a feel for math – it is taught in a very narrow way and leaves far too many feeling that math is something that other people do. In a world with more and more data, and more and more statistics/data-driven decisions this is not ok. We need everyone in business to be able to consume math intelligently, even if they can’t develop mathematical models themselves. Continuing with traditional math teaching in high school and college is just excluding most people and that has to end.

Ajay- What are the various stages of evolution that you have seen in the Decision Management Industry, including the prevailing jargon name.

James- Decision Management applications have been around for years, albeit primarily in the financial services industry. They used to all have their own categories – fraud systems, origination systems, account management systems – and this was the beginning of the category. One of the first things I did at FICO was describe all these applications as a set – recognizing that the same approach and the same cluster of technologies was being used in each case. Back in 2002 I and some colleagues started calling this approach Enterprise Decision Management. Back then most decision management was enabled by these packaged applications and the tools that could be used to build custom applications were talked about separately – business rules, optimization, predictive analytics.

Over the last 6 years the focus on decisions in each of these areas has increased – more rules people talk about managing decisions with rules and there’s more talk of improving operational decisions in predictive analytics and optimization circles. There’s more talk of using the tools/technologies together and a growing range of integrated suites/platforms.

Where most companies stand today is wanting the kind of capabilities that can only be delivered by applying decision management techniques and technologies but they are not yet asking for decision management. They want, for instance, consistent personalized offers across channels but they are not asking for centralized decision management. Based on previous experience I think this will change steadily over the next year or two with the number of companies asking explicitly for decision management capabilities rising.

From a name perspective we have evolved too. Over time it has become clear that the “Enterprise” was misinterpreted as a call for Enterprise-wide implementation of decision management when it was meant as a call from enterprise ownership of decisions. As a result some folks talk about Business Decision Management and I just like to talk about Decision Management.

Ajay- Why is Decision Management more important than say performance management, business intelligence, predictive analytics.

James- I am not sure it is more important. Most organizations need business intelligence to understand what happened in their business and they need performance management to monitor what is happening now. This kind of understanding is important in successful decision management implementations. And decision management is a management discipline designed, in part, to put predictive analytics to work in operational systems.

I do think a focus on decision is vital to all of them, however. If you don’t understand the decisions you are making it is hard for me to see how you can judge the effectiveness of either business intelligence or performance management. And predictive analytics should be even more decision-centric if it is to be effective. So a focus on decisions is a necessary prerequisite and the management of those decisions, using rules and analytics, is a great way to maximize their value in operational systems.

Ajay- What are your views on offshoring 1) High quality research 2) Labor Arbitrage technical work 3) Cost cutting driven

James- Well I think offshoring is an inevitable consequence of an interconnected world. I also think that companies that offshore simply to reduce cost deserve the employee and customer loyalty they will get as a consequence!

I do think that companies should make thoughtful decisions about what to do where, when something must be handled centrally and when it can be pushed to different localities etc. I think that smarter systems – systems that manage decisions explicitly – can help in this and help companies have a real DNA when it comes to decision making.

Ajay- What are the top 5 principles of Decision Management , as you would explain to a class of business graduates and CEO’s

James-

1. Little decisions add up

The day to day decisions that drive operational behavior, customer interactions, transactional systems are more important than the big, strategic decisions beloved of management consultants. Each one seems unimportant but they happen so often that their total value swamps anything else you do. If you get these decisions wrong it won’t matter what you get right.

2. The purpose of information is to decide

Deming has a famous quote that “The ultimate purpose of collecting the data is to provide a basis for action or a recommendation”. The reason you collect data, report on data, analyze data is to make better decisions. Otherwise it’s just a cost. And unless you know which decision you are making, and what will make it a good or a bad one, then all the data in the world (and all the data management or data analysis) will not help you.

3. You cannot afford to lock up your logic

Decision making logic- the policies, regulations, best practices and customer preferences that drive decision making – cannot be locked up in code you cannot read, systems you do not understand. No matter what else might be handled by your IT people, business decision making logic must not be. You must at least be able to collaborate with your IT folks and manage it with them. You must be responsible for this logic.

4. No answer, no matter how good, is static

Organizations must realize that they have to constantly analyze, reassess and challenge their decision making process. The effectiveness of a decision can often not be determined for some time and even a good decision can be degraded by a change in the behavior of a competitor or a change in the market. As such constant challenging of the decision making approach, constant A/B testing or adaptive control is essential if decisions are to remain effective.

5. Decision making is a process to be managed

The way you make decisions is something you must understand, document, automate and analyze. Good managers, good staff, have a good decision making process. Good outcomes might result from luck or circumstance but you don’t want to rely on that. Instead you want to focus on quality decision making processes. And like many repeatable processes, automating decision making makes it easier to analyze and improve it over time.

UPDATE-

James has agreed to schedule a free webinar to explain it more fully. Anyone who wants can register at https://decisionmanagement.omnovia.com/registration/pid=74151252469530

Ajay- What does James Taylor do when not in front of a computer, a podium or an airport. How important do you think is work life balance particularly for young people

James- Well I am a parent, a partner and an avid reader and between them those use up most of my non-work time. I really enjoy my work which makes it hard to stop sometimes. I think life/work balance is important but so is enthusiasm for what one does. Perhaps I am kidding myself but I think there is a difference between putting a lot of hours into something about which you are passionate and putting a lot of work into something just to get ahead or to avoid the rest of your life.

Ajay- Do you think BI world is male dominated. What could be the reasons.

James- Yes. The usual sexism of business combined with the average age of BI people (younger groups seem more mixed in general).

Ajay- Green economy and stimulus macro economics. How can both these fields benefit from Decision Management

James- From a macro stimulus point of view I think the key thing is that governments around the world throw money at companies specializing in decision management. <smile>

The green economy, however, is more interesting. Personally I don’t see how smart grids can be made to work without a solid core of powerful decisioning. Green marketing requires personalization and targeting to avoid waste (more decisioning) while helping consumers make better decisions about products based on green criteria needs to be built into shopping engines like Amazon’s if it is to make a real difference. Being green is all about making greener decisions and making systems make greener decisions takes decision management and decisioning technology.

Biography-

James Taylor is a leading expert in Decision Management and an independent consultant specializing in helping companies automate and improve critical decisions. Previously James was a Vice President at Fair Isaac Corporation where he developed and refined the concept of enterprise decision management or EDM. Widely credited with the invention of the term and the best known proponent of the approach, James helped create the Decision Management market and is its most passionate advocate.

James has 20 years experience in all aspects of the design, development, marketing and use of advanced technology including CASE tools, project planning and methodology tools as well as platform development in PeopleSoft’s R&D team and consulting with Ernst and Young. He has consistently worked to develop approaches, tools and platforms that others can use to build more effective information systems.

James is an active consultant, speaker and author. He is a prolific blogger, with regular posts at jtonedm.com and ebizq.net/blogs/decision_management. He also has an Expert Channel – Decision Management – on the BI Network.

His articles appear in industry magazines, he has contributed chapters to “The Business Rules Revolution:Doing Business The Right Way” (Happy About, 2006) and “Business Intelligence Implementation : Issues and Perspectives” (ICFAI University Press, 2006), and is the co-author of “Smart (Enough) Systems: How to Deliver Competitive Advantage by Automating Hidden Decisions ” (Prentice Hall, 2007) with Neil Raden .

James is a highly sought speaker, appearing frequently at industry conferences, events and seminars. He is also a lecturer at the University of California, Berkeley.

James has an M.S. in Business Systems Analysis and Design from City University, London; a B.S. in Geological Geophysics and Mathematics from the University of Reading, England; and a “Mini-MBA” certificate from the Silicon Valley Executive Business Program at San Jose State University.

You can contact James at james@jtonedm.com

Ajay- Describe your career in science from school to Senior Director in SAP to blogger/speaker. How do you think we can convince students of the benefits of learning science and maths.

Timo- I studied economics with statistics in the UK, but I had always been a closet geek and had dabbled with computers ever since I was a kid, starting with Z80 assembler code. I started my career doing low-level computer consulting in Hong Kong, and worked on a series of basic business intelligence projects at Shell in New Zealand, cobbling together a solution based on a mainframe HR system, floppy-disk transfers, and Lotus 1-2-3 macros. When I returned to Europe, I stumbled across a small French startup that provided exactly the “decision support systems” that I had been looking for, and enthusiastically joined the company.

Over the last eighteen years, I’ve worked with hundreds of companies around the world on their BI strategy and my job today is to help evangelize what works and what doesn’t, to help organizations avoid the mistakes that others have made.

When it comes to BI initiatives, I see the results of one fundamental problem almost on a daily basis: 75% of project success depends on people, process, organization, culture, and leadership, but we typically spend 92% of our time on data and technology.

BI is NOT about technology – it’s about helping people do their jobs. So when it comes to education, we need to teach our technologists more about people, not science!

Ajay- You were the 8th employee of SAP Business Objects. What are the key turning points or transition stages in the BI industry that you remember seeing in the past 18 years, and how has SAP Business objects responded to them.

Timo- Executive information systems and multidimensional databases have been around since at least the 1970s, but modern business intelligence dates from the early 1990s, driven by the widespread use of relational databases, graphical user interfaces, and the invention of the “semantic layer”, pioneered by BusinessObjects, that separated business terms from technical logic. For the first time, non-expert business people had self-service access to data.

This was followed by a period of rapid expansion, as leading vendors combined reporting, multidimensional, and dashboard approaches into fully-fledged suites. During this period, BusinessObjects acquired a series of related technology companies to complete the existing offer (such as the leader in operational reporting, Crystal Reports) and extend into enterprise information management and financial performance management.

Finally, the theme of the last few years has clearly been consolidation – according to Gartner, the top four “megavendors” (SAP, IBM, Microsoft, and Oracle) now make up almost two-thirds of the market, and accounted for fully 83% of the growth since last year. Perhaps as a result, user deployments are accelerating, with usage growth rates doubling last year.

Ajay- How do you think Business Intelligence would be affected by the following

a) Predictive Analytics.

Timo- Predictive analytics has been the “next big thing in BI” for at least a decade. It has been extremely important in some key areas, such as fraud detection, but the dream of “no longer managing by looking out of the rear-view mirror” has proved hard to achieve, notably because business conditions are forever changing.

We offer predictive analytics with our Predictive Workbench product – but I think the real opportunity for this technology in the future is “power analytics”, rather than “prediction”. For example, helping business people automatically cluster similar values, spot outliers, determine causal factors, and detect trend inflection points, using the data that they already have access to with traditional BI.

b) Cloud Computing.

Timo- In terms of architecture, it’s clearly not about on-demand OR on-premise: it’s about having a flexible approach that combines both approaches. You can compare information to money: today, we tend to keep our money in the bank rather than under our own mattress, because it’s safer, more convenient, and more cost-efficient. At the same time, there are situations where the convenience of cash is still essential.

Companies should be able to choose a BI strategy, and decide how to deploy it later. This is what we offer with our BI on-demand solutions, which use the same technology as on-premise. You can start to build on-premise and move it to on-demand, or vice-versa, or have a mix of both.

In terms of data, “cloud intelligence” is still a work in progress. As with modern financial instruments, we can expect to see the growth of new information services, such as our “information on-demand” product that provide data feeds from Reuters, Thompson Financial, and other providers to augment internal information systems. Looking further into the future, we can imagine new information marketplaces that would pay us “interest” to store our data in the cloud, where it can be adapted, aggregated and sold to others.

c) Social Media.

Timo- Conversations and collaboration are an essential part of effective business intelligence. We often talk about the notion of a “single view of the truth” in this industry, but that’s like saying we can have “a single view of politics” – while it’s vital to try to give everybody access to the same data, there will always be plenty of room for interpretation and discussion. BI platforms need to support this collaborative decision-making.

In particular, there are many, many studies that show up our all-too-human limitations when it comes to analyzing data. For example, did you know that children with bigger feet have better handwriting?

It’s absolutely true — because the children are older! Mixing up correlation and causality is a common issue in business intelligence, and one answer to the problem is to add more people: the more reviewers there are of the decision-making process, the better the decisions will be.

Analysis is also critical to the development of social media, such as analyzing sentiment trends in Twitter — a functionality we offer with SAP CRM — or tracking social communities. For example, Jive, the leader in Enterprise 2.0 platforms, offers our BI products as part of their solution, to help their customers analyze and optimize use of the system. Administrators can track if usage is trailing off in a particular department, for example.

d) Social Network Analysis.

Timo- Over the last twenty years, partly as a result of extensive automation of operational tasks with systems such as SAP, there’s has been a huge shift from “routine” to “non-routine” work. Today, fully 90% of business users say that their work involves decision making, problem solving, and the creation of new analysis and insight.

To help support this new creativity, organizations are becoming more porous as we work closer with our ecosystem of customers, partners, and suppliers, and we work in ever-more matrixed environments and cross-functional teams.

We’ve developed a Social Network Analyzer prototype that combines BI and social networking to create a “single view of relationships”. It can gather information from multiple different systems, such as HR, CRM, email distribution lists, project teams, Twitter, etc., to create a multi-layered view of how people are connected, across and beyond the enterprise. For more information, see the SAP Web 2.0 blog post, and you can try it yourself on our ondemand.com web site.

Ajay- What is the area that SAP BusinessObjects is very good at (strength). What are the key areas that you are currently seeking to improve ( opportunities)

Timo- Companies evaluating BI solutions should look at four things: product functionality for their users’ needs, fit with the overall IT architecture, the vendor’s reputation and ecosystem, and (of course) price. SAP BusinessObjects is the clear leader in the BI industry, and I’d say that SAP BusinessObjects has the best overall solution if you’re a large organization (or looking to become one) with a variety of user needs, multiple data sources, and a heterogeneous IT infrastructure.

In terms of opportunities, we have high expectations for new interfaces for casual users, and in-memory processing, which we have combined in our SAP BusinessObjects Explorer product. Initial customer feedback has been excellent, with quotes such as “finding information is as easy as using the internet” and “if you can use a computer, you can use Explorer”.

In terms of future directions, we’re taking a very transparent, Web 2.0 approach. The SAP Business Objects innovation center is modeled on Google Labs and we share our prototypes (including the Social Network Analyzer mentioned above) with anybody who’s interested, and let our customers give us early feedback on what directions we should go.

Ajay- What does Timo Elliott do for work life balance when not writing, talking, and evangelizing about Business Intelligence?

Timo- I’m a keen amateur photographer – see www.timoelliott.com/personal for more!

Biography- http://timoelliott.com/blog/about

Timo Elliott is Senior Director of Strategic Marketing for SAP BusinessObjects. For the last twenty years he has been a thought leader and conference speaker in business intelligence and performance management.

A popular and engaging speaker, Elliott presents regularly to IT and business audiences at international conferences, drawing on his experience working with enterprise customers around the globe. Topics include the latest developments in BI/PM technology, how best to suceed with BI/PM projects, and future trends in the industry. 

Prior to Business Objects, Elliott was a computer consultant in Hong Kong and led analytics projects for Shell in New Zealand. He holds a first-class honors degree in Economics with Statistics from Bristol University, England.

Additional websites: http://www.sapweb20.com —  web 2.0 technology by, with, and at SAP

Email: telliott@timoelliott.com or timo.elliott@sap.com

LinkedIn: http://www.linkedin.com/in/timoelliott

Twitter: http://twitter.com/timoelliott

Flickr: http://www.flickr.com/photos/timoelliott/

Facebook: http://www.facebook.com/people/Timo-Elliott/544744135

For an earlier interview with Oracle Data Mining Product Management, Charlie Berger see http://decisionstats.wordpress.com/2009/09/02/oracle/

Ajay- Describe your career in science from your high school days to the science books you have written. What do you think can be done to increase interest in science in young people.

John Fox- I’m a sociologist and social statistician, so I don’t have a career in science, as that term is generally understood. I was interested in science as a child, however: I attended a science high school in New York City (Brooklyn Tech), and when I began university in 1964 at New York’s City College, I started in engineering. I moved subsequently through majors in philosophy and psychology, before finishing in sociology — had I not graduated in 1968 I probably would have moved on to something else. I took a statistics course during my last year as an undergraduate and found it fascinating. I enrolled in the sociology graduate program at the University of Michigan, where I specialized in social psychology and demography, and finished with a PhD in 1972 when I was 24 years old. I became interested in computers during my first year in graduate school, where I initially learned to program in Fortran. I also took quite a few courses in statistics and math.

I haven’t written any science books, but I have written and edited a number of books on social statistics, including, most recently, Applied Regression Analysis and Generalized Linear Models, Second Edition (Sage, 2008).

I’m afraid that I don’t know how to interest young people in science. Science seemed intrinsically interesting to me when I was young, and still does.

Ajay- What prompted you to R Commander. How would you describe R Commander as a tool, say for a user of other languages and who want to learn R, but get afraid of the syntax.

John- I originally programmed the R Commander so that I could use R to teach introductory statistics courses to sociology undergraduates. I previously taught this course with Minitab or SPSS, which were programs that I never used for my own work. I waited for someone to come up with a simple, portable, easily installed point-and-click interface to R, but nothing appeared on the horizon, and so I decided to give it a try myself.

I suppose that the R Commander can ease users into writing commands, inasmuch as the commands are displayed, but I suspect that most users don’t look at them. I think that serious prospective users of R should be encouraged to use the command-line interface along with a script editor of some sort. I wouldn’t exaggerate the difficulty of learning R: I came to R — actually S then — after having programmed in perhaps a dozen other languages, most recently at that point Lisp, and found the S language particularly easy to pick up.

Ajay- I particularly like the R Cmdr plugins. Is it possible for anyone to increase R Commander with a customized package- plugin.

John- That’s the basic idea, though the plug-in author has to be able to program in R and must learn a little Tcl/Tk.

Ajay- Have you thought of using the R Commander GUI on an Amazon EC2 and thus making R high performance computing say available on demand ( similar to Zementis model deployment using Amazon Ec2). What are you views on the future of statistical computing

John- I’m not sure whether or how an interface like the Rcmdr, which is Tcl/Tk-based, can be adapted to cloud computing. I also don’t feel qualified to predict the future of statistical computing.

I think that R is where the action is for the near future.

Ajay-What are the best ways for using R Commander as a teaching tool ( I noticed the help is a bit outdated).

John- Is the help a bit outdated? My intention is that the R Commander should be largely self-explanatory. Most people know how to use point-and-click interfaces. In the basic courses for which it is principally designed, my goals are to teach the essential ideas of statistical reasoning and some skills in data analysis. In this kind of course, statistical software should facilitate the basic goals of the course.

As I said, for serious data analysis, I believe that it’s a good idea to encourage use of the command-line interface.

Ajay- What are your views on R being recognized by SAS Institute for it’s IML product. Do you think there can be a middle way for open source and proprietary software to exist.

John- I imagine that R is a challenge for producers of proprietary software like SAS, partly because R development moves more quickly, but also because R is giving away something that SAS and other vendors of proprietary statistical software are selling. For example, I once used SAS quite a bit but don’t anymore. I also have the sense that for some time SAS has directed its energies more toward business uses of its software than toward purely statistical applications.

Ajay- Do people in R Core team recognize the importance of GUI? What does the rest of R community feel? What has the feedback of users ben to you. Any plans to corporate sponsors for R Commander ( Rattle , an R language data mining GUI has a version called Rstat at http://www.informationbuilders.com/products/webfocus/predictivemodeling.html while the free version and code is at rattle.togaware.com)

John- I feel that the R Commander GUI has been generally positively received, both by members of R Core who have said something about it to me and by others in the R community. Of course, a nice feature of the R package system is that people can simply ignore packages in which they have no interest. I noticed recently that a Journal of Statistical Software paper that I wrote several years ago on the Rcmdr package has been downloaded nearly 35,000 times.

Because I wouldn’t expect many students using the Rcmdr package in a course to read that paper, I expect that the package is being used fairly widely.

Ajay- What does John Fox do for fun or as a hobby?

John- I’m tempted to say that much of my work is fun — particularly doing research, writing programs, and writing papers and books. I used to be quite a serious photographer, but I haven’t done that in years, and the technology of photography has changed a great deal. I run and swim for exercise, but that’s not really fun. I like to read and to travel, but who doesn’t?

Biography-

Prof John Fox is a giant in his chosen fields and has edited/authored 13 books and written chapters for 12 more books. He has also written and been published in almost 49 Journal articles. He is also editor in chief for R News newsletter. You can read more about Dr Fox at http://socserv.mcmaster.ca/jfox/

On R Cmdr-

R Cmdr has substantially decreased the hygiene factor for people wanting to learn R- they begin with the GUI and then later transition to customization using command line. It is so simple in its design that even under graduates have started basic data analysis with R Cmdr after just a class.You can read more on it here at http://socserv.mcmaster.ca/jfox/Misc/Rcmdr/Getting-Started-with-the-Rcmdr.pdf

There will always be lawyers and financiers who make loads of money. But they will have quantitative experts on their teams- Stephen L Baker

Ajay- Describe your career journey from high school to a technology writer to author of The Numerati.

Steve- I was always interested in history and in literature, and in college I fell in love with Spanish. So after college, I moved to Ecuador, taught English, and wrote fiction. I saw early on that I wasn’t going to be able to make a living with fiction. So I went into journalism. My goal was to become a correspondent in Latin America. Through my 20s, I worked in Vermont, Madrid, Argentina, Venezuela, Washington DC, and El Paso, Texas. And I finally got the job I was looking for, bureau chief for the Mexico bureau of BusinessWeek magazine.

After Mexico, my family and I moved to Pittsburgh. It appeared that the magazine was losing interest in heavy industry in the mid-90s, so I began to write about software and robotics coming out of at Carnegie Mellon University. That was my transition into technology. A year later, BusinessWeek offered me a job covering technology in Europe. I moved with my family to Paris, where we lived for four years. I focused largely on mobile communications. It seemed to me that the combination of mobility and the Internet would fundamentally change communications.

I returned to New York in 2002. I focused on big picture stories. One day in 2005, I proposed a story about the decline of the U.S. technology industry. I argued that we were behind in wireless and in broadband, we were graduating fewer scientists and engineers than other great powers, especially in Asia. One editor pointed out that mathematics was critical for these competitive issues. The editor in chief, Steve Adler, called for a cover story on math, and he assigned it to me. I didn’t know much about math at the time, and I still don’t. But this gave me the chance to dive into the world of data analysis. I wrote a cover story, Math will Rock Your Business, and later got the contract to write the Numerati.
Ajay- How do you think the government can motivate more American students to science careers?

Steve- I think focusing on the science that kids find cool–robotics, space and ocean exploration, would help. Funding basic research would be useful. But I don’t think it’s entirely a governmental issue. Parents, companies, universities, they all have to participate.
Ajay- What are the top  tips you would give to aspiring technology writers and bloggers (like myself)?

Steve-
1) Learn about non tech subjects, such as history, literature, art and psychology
2) Work on writing clearly for non experts. Avoid jargon.
3) Do reporting
4) Do more reporting

Ajay-The Numerati portrays a math elite which breaks the stereotype of the lonely, nerdy geek. How important do you think is that common people be more educated in math so they are more aware of marketing operations and credit offers?

Steve- I think it’s important for common people, as you call them, to understand basic statistics. More and more of our lives are going to be analyzed and communicated to us statistically. Those who do not understand this will not know to ask the right questions, and will be easily fooled. This is also true within companies. CEOs can be fooled by numbers, just like anyone else.
Ajay- Asia delivers a disproportionate number of science graduates. Yet one generation ago American and European heritage scientists made the trip to the moon with very basic computers. As our lives get increasingly shaped by the Numerati, how important are geo-cultural influences in its membership?

Steve- Most of the Numerati I met in the United States were born outside the U.S. The US has long relied on foreign brains, especially for its technology industry. As the Numerati study people’s lives, the quantitative experts will increasingly need to work closely with linguists, anthropologists, and psychologists. And they’ll need to understand different global cultures and languages. In this sense, the international nature of the Numerati is an advantage.
Ajay- Do you think the shift in money and influence from lawyers and financiers to scientists and mathematicians is temporary or is it here to stay?

Steve-I think it’s here to stay. There will always be lawyers and financiers who make loads of money. But they will have quantitative experts on their teams.

Ajay- What influenced your decision to be associated with Predictive Analytics world?

Steve- I had the privilege of interviewing Eric Siegel as I was researching the book. We’ve kept in touch since then. I think he’s very bright and does excellent work.
Ajay- What does Stephen Baker do when not writing books or articles or observing the world go around him?

Steve- I like to ride bicycles, I like to travel. I love Spanish and French and baseball and music

Biography-

Stephen L. Baker is the author of The Numerati and a senior writer at BusinessWeek, covering technology. Previously he was a Paris correspondent. Baker joined BusinessWeek in March, 1987, as manager of the Mexico City bureau, where he was responsible for covering Mexico and Latin America. He was named Pittsburgh bureau manager in 1992. Before BusinessWeek, Baker was a reporter for the El Paso Herald-Post. Prior to that, he was chief economic reporter for The Daily Journal in Caracas, Venezuela. Baker holds a bachelor’s degree from the University of Wisconsin and a master’s from the Columbia University Graduate School of Journalism.

You can read more about the Numerati at http://thenumerati.net/index.cfm?catID=18 Stephen L Baker is the keynote speaker at Predictive Analytics World and you can check the details here http://www.predictiveanalyticsworld.com/register.php if you want to listen to  him at the event.

You can follow Steve on twitter at http://twitter.com/stevebaker and follow his blog here http://www.businessweek.com/the_thread/blogspotting/

If we don’t learn from history, we are condemned to repeat it.

Ajay- Describe your career in science. How would you motivate children in class rooms today to be as excited about science as the moon generation was?

J Bass- My graduate training was in economics and statistics.  I have used that training in ways that I would never have anticipated when I was in graduate school 30 years ago.  But it is still exciting for me.  I started out building microeconomic models, then went on to write statistical language compliers and build health policy macroeconomic models.  These days I develop and articulate health policy to help increase patient’s access to cutting edge medicines.  The company I work for now is very science based and even applies scientific thinking, measurement and testing of alternatives in the business side of its operations.

I spend volunteer time as a guest teacher at local middle schools, high schools and community colleges.  I often talk about math and statistics and have found that one way to help motivate students is to give them “fun” example problems.  I often use an example of the 1969 lunar orbital calculations to motivate basic trigonometry and quite a number of students who say they don’t like math end up loving solving parts of that problem.  I think our school curriculums need to come up with problems and examples that the students find interesting.  I’m not sure our existing curricula processes make this an easy thing to do.  All too often we teach techniques without combining that teaching with strong motivating examples that make learning fun.

Ajay-  What are the changes in paradigms that you have seen across the decades? What are the key insights and summaries that you can provide.

J Bass- Our increasing understanding of biology and DNA is a major paradigm shift that is combining molecular biology and protein chemistry with computer science.  Identifying the human DNA sequence was only the beginning.  Imagine that you were handed the bit sequence of a CD-ROM and were told to figure out what parts of it were a text document, what parts were a JPEG photograph and what parts were an MP3 music file – if you did NOT know the coding schemes of such files.  That’s analogous to where we are today with DNA sequences…we know the ATCG sequence, but we are only scratching the surface of understanding the things that the DNA sequence codes for – proteins, cell metabolism, differentiating cell reproduction. It is an exciting time where multidisciplinary convergences are going to cause big leaps in our understanding of the way the world works.  Our traditional undergraduate and graduate education programs tend to provide specialists rather than generalists.  I think that the coming decades will need a new breed of generalists who can bridge the learning that is taking place in diverse areas.

Ajay-  Open source is just a flash in the pan- ultimately someone has to pay the rent for the programmers. Comment please.

J Bass- I think that open source is here to stay and is an important driving force behind continuing software innovation.  Charging for software licenses is not the only way to reward programmers.  Selling consulting and support services, or add on software often seems to pay well enough to keep the open source model thriving.  The Linux kernel project is run by volunteers (and still led by Linus Torvalds after many years and multiple job changes) and many of these volunteers are salaried programmers and engineers for software firms that benefit from open source.  Quite a few of the open source volunteers are working on their open source project as part of their “day job” responsibilities.  Some open source projects– R, for example — seem to have a lot of the time spent by their contributors being related to their “day job”, like academic statisticians who are trying to code and test new algorithms as part of their published research.  Open source often becomes a big part of important commercial products as well.  I personally use Apple computers and the Mac OS is a wonderful product developed at least in part from BSD Unix open source roots.  There is a Mac OS X related open source operating system called Darwin that is supported in part by Apple staff programmers.  The synergies between open source development and commercial software product development are widespread and likely to continue.  I have been a supporter of the Free Software Foundation (see www.fsf.org) for years and I think they are doing good work by keeping the “free software” vs. “open source” controversy alive.

Ajay-  What does Jeff Bass do in his spare time today?

J Bass- I am involved in charity work more and more as I get older.  I am on the National Board of Directors of the Arthritis Foundation and I help that organization to raise money for patient education, camps for kids with arthritis and disease research.  One way I raise funds is by participating in an 8 day,  525 mile bicycle tour – the Arthritis Foundation California Coast Classic – to raise funds and awareness for arthritis programs and research.  I have ridden that tour from San Francisco to Los Angeles 7 times and it has turned bicycling into a year round hobby and fitness regimen for me.  Not something I ever would have expected myself to be involved in (you can visit www.cccsocal.kintera.org/bass to learn more).  I have also been working on my private pilot’s license and am I am working to design and build an electric airplane using lithium polymer batteries.  I have a large array of solar panels on the roof of my house and I drive an electric car powered by them.  I hope to one day fly a plane powered by them as well.

This is part two of the interview with the founder of Bass Institute, Jeff Bass. You can see part of the interview here http://www.decisionstats.com/2009/08/28/software-history-bass-institute-part-1/

The history of BI pretty much follows the history of computing platforms. First we had time-sharing, then mainframes, then mini’s, then client-server vs. PC, then a number passes at distributed computing, such as CORBA, then SOA and now the cloud.- Neil Raden

Ajay- Describe your career in math and technology and your current activities. How would you explain what you do for a living to a group of high school students who are wondering to take up mathematical and technical subjects or not.

Neil- I didn’t earn a dime at the career I was meant for, consulting, until I was 33 years old. So I would tell college students not to be in such a hurry to corner themselves into a career. It may take a while to figure out what you really want.

Though I went to college to study theatre, within a few weeks I was inspired by a math professor and switched my major. From that point on, it was pads of paper and sharp pencils. I was totally in my own head with math. I never took a statistics course, or even differential equations, because I was consumed by discrete math (graph theory too), topology and logic and later game theory/economics.

When I went looking for a job in 1974, in the midst of a deep recession, I was confronted with the stark reality (in New York ) that I could be a COBOL programmer or an actuary. I chose the latter. Working at AIG in New York in the 70’s was pretty exciting. We broke new ground in commercial property and casualty insurance and reinsurance every week. I was part of a small R&D group under the chief actuary, who reported directed to Maurice Greenberg, the legendary (but now maligned) inventor of AIG, and I loved the work.

I had to go back and teach myself probability and statistics to get through the exams, but ultimately, two kids and one on the way in NYC on one not-so-great salary was a deal-breaker. I left AIG and joined a software company doing modeling and prediction. The rest, as they say, is history. I formed my own consulting company in 1985 and I’m still at it.

To me, consulting isn’t something you do between jobs or a title you get because you implement software for clients. Consulting is a craft, it’s a career and it is rather easy to do but very difficult to learn. I work very hard to teach this to people who work for me. It’s about commitment, hard work and, most of all, ethics and being authentic with your client.

Ajay- Writing books is a lonely yet rewarding work. Could you briefly elucidate on your recent book, Smart (Enough) Systems?

Neil- I have to credit my partner, James Taylor, with the concept for the book. He was working at Fair Isaac (now FICO) at the time and this was exactly what he was doing there. It was a little tangential to my work, but when James approached me, he said he wanted a partner who was proficient in the data integration and analytics aspects of EDM (Enterprise Decision Management).

James made it pretty easy because

1) he is very prolific and 2) he took most of my comments and integrated them without argument.

I’d say I was pretty lucky and it went very well. I don’t know if I’ll ever write another book. I suppose I won’t know until the idea hits me. I’m sure it will be more difficult doing it on my own.

Ajay- What are the various stages that you have seen the BI industry go through. What are the next few years going to bring to us-

What is your wishlist for changes the industry makes for better customer ROI.

Neil- The history of BI pretty much follows the history of computing platforms. First we had time-sharing, then mainframes, then mini’s, then client-server vs. PC, then a number passes at distributed computing, such as CORBA, then SOA and now the cloud. But while the locus of BI storage, computing and presentation has changed, it’s focus changes very slowly.

Historically, there have been two major subject areas in BI: f inance and sales/marketing, All of the other subject areas still rest on periphery.

Complex Event Processing ( CEP ) for example, is making a lot of noise lately , but not much implementation. Visualization is here to stay . When the BI app and the Web a pp are the same, BI will be everywhere, but it will be a sort of pyrrhic victory because it won’t be recognized as such. Now you can take all of this with a grain of salt because I don’t really follow the industry per se, I’m more interested in how my clients can apply the technology to get the results they need.

Ajay- There is a lot of buzz about predictive analytics lately. Do you think it will have a noticeable impact or is it just the latest thing?

Neil- There are only so many people who understand quant itative meth ods and it isn’t going to grow very much. This puts a damper on PA (Predictive Analytics) because no manager is going to act on the recommendations of a black box without an articulate quant who can explain the methodology and the limits of its precision.

That isn’t a bad thing, and those who practice in predictive analytics will prosper.

On the other hand, I believe there will be an expansion of the use of generic PA models that have been vetted in practice. The FICO score is a good example, and the ability to develop and implement these applications (it’s much easier now thanks to PA software and computing environments in general) should allow for a nice market to develop around them. This is especially true with decision automation systems, like logistics, material handling, credit authorization, etc.

Ajay- What were your most interesting projects as an implementer? Most rewarding?

Neil- Most Interesting: I was the Chairman of an Advisory Board at Sandia National Laboratories for a few years.Our goal was to encourage the lab to adopt more modern and effective information management tools for their dual purpose of

1) designing and manufacturing nuclear weapons (frightening isn’t it?) and

2) certification of nuclear waste repositories.

I was able to work with scientists, physicists, engineers, geologists and computer sciences, all from backgrounds very different from those I normally engaged. The problems were monumental.

Most rewarding: We developed a data warehouse to capture the daily sales of products at the most detailed level for a cosmetics company. They never had this information before because the retailers were counters in hundreds of department stores. Thus they were able for the first time to truly understand the “sell through” of their products. Beyond just allowing a better understanding of the flow, they could tailor their promotions and, not much later, implement a continuous replenishment system.

The president of the company came to the launch and explained how we had allowed the company to do things it had never done before which would change it for the better. You don’t get those accolades from the CEO very often.

Ajay- You’ve written forty white papers. That’s a lot. What impact do you think they’ve had?

Neil- I couldn’t tell you. I don’t track downloads, my website doesn’t even require registration. I don’t see them quoted or cited very often, but then, people don’t quote or cite other’s work in this field very often anyway. I can say that I have many repeat customers among the vendors, so they must be deriving some value from them.

Ajay- What are your views on creating a community for the top 100 BI analysts in the world – a bit like a Reuters or a partnership firm. How pleased do you think will BI vendors be by this.

Neil- I was actually involved in an effort like this about a dozen years ago, called BI Alliance . Doug Hackney and I started it, and we had about a dozen BI luminaries in the organization. I’ll try to remember some: Sid Adelman, David Marco, Richard Winter, David Foote, Herb Edelstein.

You could only join if you were an independent or the head of your own firm.

It was a useful marketing tool as we were able to 1) share references and 2) staff projects. But it sort of lost its inertia after a few years.

But a few hundred BI analysts? Are there that many?? LOL I don’t know how the vendors would react, but I sort of doubt this sort of organization would have any kind of clout – too many divergent opinions.

Ajay- Do you think the work you do matters?

Neil- It certainly has an economic impact on my family! LOL I don’t know, I hope it does and proportionate to my income versus the size of the industry, yes, I guess it does. Not necessarily directly though .

A company in Dayton or Macon doesn’t make a decision because I said so, but I think I do influence some analysts and vendor s a nd to the extent I influence them, then I guess I do . I limit my analysis to my clients. If they think this work matters, then it does.

Biography-

Neil Raden, consultant, analyst and author is followed by technology providers, consultants and even other analysts. His knowledge of the analytical applications is the result of thirty years of intensive work. He is the founder of Hired Brains, a research and advisory firm in Santa Barbara, CA, offering research and analysis services to technology providers as well as providing consulting and implementation services. Mr. Raden began his career as a casualty actuary with AIG before moving into software engineering and consulting in the application of analytics in fields as diverse as health care to nuclear waste management to cosmetics marketing. His blog can be found at intelligententerprise.com/experts/raden/. He is the author of dozens articles and white papers and he has has contributed to numerous books and is the co-author of “Smart (Enough) Systems” (Prentice Hall, 2007) with James Taylor. nraden@hiredbrains.com

Alternatively you can just follow Neil Raden at his twitter id neilraden

Ajay- Describe your career in science and technology. What do you think is the best thing that science careers offer to people.

Merv- I wouldn’t characterize myself as having worked in science – even computer science implies a direction quite different from my own. I began as a statistician, and after getting the opportunity to learn some computer skills, spent a number of years coding a variety of decision support and data integration programs. I joined the software industry as a technical journal editor, and held a variety of marketing, strategy and analyst relations positions before beoming an industry analyst.

Ajay-  With your background in finance, how do you think the next generation of financial reporting systems should be built for early warning signals of crisis. Due to think predictive analytics can play a bigger role than just traditional reporting and metric aggregation.

Merv- We’re already seeing greater specialization in financial applications that provide context: an understanding of the industry the firm is in, and its special requirements for industry standards, compliance, etc. This added specific depth and breadth, combined with increasing sophistication in pre-built models for predictive analytics and access to hitherto unavailable volumes of historical data, will extend the reach of applications used by financial professionals. Guided analysis and advanced visualization will make more sophisticated tools available and understandable, promoting more awareness of impending problems and recommending courses of action.

Ajay- What tips would you like to give to aspiring analysts or science journalists and bloggers. What are the top 5 things do’s /don’ts that you refer to while writing a report or analysis.

Merv-

* Never stop learning, and never assume you know enough. Be humble enough to acknowledge that the person you talk to may have something to teach you – and ask about anything you hear that you don’t understand.

* Remember your audience. If you don’t know who you’re writing for, how can you decide what matters to them?

* Explain why what you’re saying matters, and to whom. Don’t assume your readers know.

* Be clear about what is new or changed. If it’s business as usual, there should be a good reason to write about it – maybe change was expected but is slow in coming.

* Acknowledge your sources and collaborators. Be generous with credit.

Ajay- Recently some BI Analyst firms saw the departure of star analysts to found their own firm. How do you think tech companies can manage the retention of talented people especially those who become a bigger brand than they were originally supposed to.

Merv- I’m assuming you want the response to refer to analyst firms. “Bigger than they were originally supposed to” is not an accurate way to describe the sitution.

Analyst firms don’t put a ceiling on their employees’ brand building; quite the contrary. They train them, give them a platform, and sustain them as they do so. Nonetheless, the firm’s brand, not the individual analyst’s brand, is what matters to them.

In general, the big ones don’t care about retaining people. They believe they can easily replace them, and history shows that they recover well from such departures.

Ajay- What incentives apart from the usual financial ones can help build a culture of intrapreneurship in which employees help build startups within the parent firm.

Merv- You can’t leave finances out of it. The business model that has been shown to work for intrapreneurs is partnership, where the partners share significantly in successful practices they build and deliver.

In consulting firms, if you build a practice, you benefit from its success. Analyst firms are increasingly making consulting part of the analyst job description, but the big ones have not made any moves to institute a partner-style model. So to your point, those who build their own brand successfully are likely to leave.

They become entrepreneurs, not intrapreneurs.

Ajay- What are your views on the next 12 months in terms of technology and BI industry dynamics. What is your wishlist- the top three things that you wish happen in the field of technology.

Merv-

* We’re entering a period of great ferment in data management as a set of upstarts has had early success with specialized analytic database platforms. As spending rebounds, most will see their momentum continue. Several have new funding, strong management teams, and early successes to build on.

* The rest of BI will see similar expansion. BI is a perennial growth market and it’s not about to slow down – predictive analytics, advanced visualization, more spohisticated and widespread use of text analytics, and the movement to SaaS models will play a role.

* True analytic applications, as described above in a financial context, will also continue their momentum. You’ll see them in other places, from manufacturing to retail, as micro-verticalization and the proliferation of templated best business practice models roll out form the largest players.

Those are both predictions and a wish list – they move us closer to delivering on the promise of having computing power help us improve business results. There are many more changes coming in packaging, licensing, the emergence of dramatically more powerful hardware platforms – but that’s business as usual. The aging server population will need replacement, and the rebound will be substantial, hastening the generational shift. And a skills shortage will re-accelerate the growth of offshoring. The next decade will be transformational in many ways.

Ajay- How does Merv Adrian balance his work and home life? How important is work life balance in this profession and do you think younger analysts sometimes dont pay attention to it.

Merv- As a manager, I always did my best to remind analysts working for me to leave time for the things that define us a humans – family, friends and faith. What we leave behind will be there, not in our reports, no matter how good they are. We all find ourselves consumed by our work, and social media exacerbates the situation unless we build in ways to be human too.

Youth will always be in a hurry, but the natural maturation process usually works out fine. Where I see balance begin to reassert itself is usually in the family – when your kids arrive, you must stop and remember to be there for them. You mustn’t delegate that one – nobody has ever looked back at their life and said “I wish I had spent less time with my kids.”

Thanks for asking, Ajay. My final thought is that all this guidance is aspirational; I struggle for balance every day. Sometimes I do pretty well at it,and often I fall short. I try to keep my values firmly in mind and strive to live up to them, as we all do.

–

Analyst and consultant Merv Adrian founded IT Market Strategy after three decades in the IT industry. During his tenure as Senior Vice President at Forrester Research, he was responsible for all of Forrester’s technology research, and covered the software industry. Earlier, as Vice President at Giga Information Group, Merv focused on facilitating collaborative research and covered data management and middleware. Prior to becoming an analyst, Merv was Senior Director, Strategic Marketing at Sybase, where he also held director positions in data warehouse marketing and analyst relations

For more on Merv’s views you can go here

Merv’s B-Eye Network channel at http://www.b-eye-network.com/channels/5097/
Blog: http://mervadrian.wordpress.com
Twitter: merv

Ajay- Describe your career in data mining so far from college, jobs, assignments and projects. How would you convince high school students to take up science careers?

Charlie- In my family, we were all encouraged to pursue science and technical fields. My Dad was a Mechanical Engineer and all my siblings are in scientific and medical fields. Early on, I had narrowed my career choices to engineering or medicine; the question when I left for college was which kind. My Freshman Engineering exposed students to 6 weeks of the curriculum for each of the engineering disciplines. I found myself drawn to the field of Operations Research and Industrial Engineering. I liked the applied math and problem solving aspects. While not everyone has an aptitude or an interest in Math or the Sciences, if you do, it can be a fascinating field.

Ajay- Please tell us some technical stuff about Oracle Data Mining and Oracle Data Miner products. How do they compare with other products notably from SAS and SPSS? What is unique in Oracle’s suite of data mining products- and some market share numbers to back these please?

Charlie- Oracle doesn’t share product level revenue numbers. I can say that Oracle is changing the analytics industry. Ten years ago, when Oracle acquired the assets of Thinking Machines, we shared a vision that over time, as the volumes of data expand, at some point, you reach a point where you have to ask whether it makes more sense to “move the data to the algorithms” or to “move the algorithms to the data”. Obviously, you can see the direction that Oracle pursued. Now after 10 years of investing in in-database analytics, we have 50+ statistical techniques and 12 machine learning algorithms running natively inside the kernel of the Oracle Database. Essentially, we have transformed the database to become an analytical database. Today, you now see the traditional statistical software vendors announcing partnering initiatives for in-database processing or in the case of IBM, acquiring SPSS. Oracle pioneered the concept of using a relational database to not only store data, but to analyze it too. Moving forward, I think that we are close to the tipping point where in-database analytics are accepted as the winning IT architecture.

This trend towards moving the analytics to where the data are stored makes a lot of sense for many reasons. First, you don’t have to move the data. You don’t have to have copies of the data in external analytical sandboxes where it open to security risks and over time, becomes more aged and irrelevant.

I know of one major e-tailor who constantly experiments by randomly showing web visitors either offers “A” or a new experimental offer “B”. They would export massive amounts of data to SAS afterwards to perform simple statistical analyses. First, they would calculate the median purchase amounts for the duration of the experiment for customers who were shown both offers. Then, they would perform a t-test hypothesis test to determine whether a statistically valid monetary advantage could be gained. If offer “B” were outperforming offer “A”, the e-tailor would convert the web store to show everyone the “better” offer.

Over the past 10 years, we’ve added 50+ of the most commonly used statistical functions as base features of every Oracle Database. Now “A/B offer” experimentation can be performed 100 percent inside the Oracle Database.

Ajay- What areas, or data conditions would you not recommend Oracle Data Miner to a customer?

Charlie- If you don’t already have any database expertise in your organization, have modest amounts of data or you are doing a simple “one-off” analysis, Oracle Data Mining may be too much horsepower for your problem. However, if you were college student learning technical skills, I would learn how to do data analysis in the database. That is where everyone will be doing it in the future.

Ajay- With the slew of Oracle Applications, will we be seeing any data mining appearing in these applications?

Charlie- Yes. You are already seeing it. Oracle announced the CRM OnDemand Sales Prospector application that integrates Oracle Data Mining as a recommendation engine inside the software designed for sales reps. As we already have the data and we know in what context the user wants to use that data, we’ve built automated data mining into Sales Prospector that identifies those prospects and which products have the highest likelihood of closing. We provide the expected length of time to close the deal, the size of the deal, and even offer likely references that should be helpful in closing the sale.

At Oracle Open World (OOW), we have many tens of thousands of customers join us for a week in San Francisco that is filled with hundreds of technical talks that span a wide range of technical topics, product features, industries and user roles. We embed Oracle Data Mining in the Oracle Open World Schedule Builder and mine the abstract text and attendee profiles to find patterns. As Oracle Open World attendees plan their week’s schedule, we suggest recommendations for the Oracle Open World talks in which they are most likely to be interested.

You can imagine a wide range of applications within Oracle where cleverly factory installing a bit of predictive analytics could provide great value to a user, a manager, a decision maker or a customer. Imagine being able to have insight into which employees are most likely to voluntarily leave, which expense report submissions might be anomalous, which products customers are most likely to be interested in, insights how to make your IT platform perform better.

The problem in the past has been that each application is different and different vendors were involved with different aspects of any total solution. If you have the data and know the use cases, that makes it a lot easier. Now that Oracle is a major player in everything from the database, to middleware, to applications and industry verticals, it is logical to anticipate customer use cases and build solutions that leverage the data and predictive analytics—too add greater value. I am a data miner; everywhere I look I see opportunities to automatically the mine data, gain new insights, and distribute that new information to where and when it is most needed to put that information to competitive gain.

Ajay- Describe some work that Oracle has been doing in predictive analytics, text mining and forecasting algorithms.

Charlie- Oracle started on this journey over 10 years ago. Our first data mining algorithms were the Naïve Bayes classification and A Priori (market basket analysis) algorithms. We did this because it made the most sense at the time. The database is good at counting things, and these algorithms were based on “simple” conditional probabilities. Overtime, we extended the database to add new supporting analytical features and expanded our set of machine learning algorithms and capabilities to include clustering, decision trees, support vector machines, special cases for anomaly detection, and the ability to mine text or unstructured data. As the Oracle Data Mining algorithms run in the kernel of the database and can mine tables, views, they also take advantage of all the vast capabilities of the Oracle Database including Partitioning, Security, Text, Real Application Clusters and now Exadata, Oracle’s new offering that combines software and hardware to offer incredible performance and scalability. In all cases, Oracle Data Mining is just another core capability of that same Database so everything is kept simple for IT. Whether you are looking to mine a single table or build a sophisticated application that automatically mines star schema data form a variety of sources and deploys the models and results in real-time applications, it is all built on the same IT, albeit now more analytically capable, platform. In the just released Oracle Database 11g Release 2, Oracle Data Mining models are pushed to the Exadata storage layer for significant (several multiples to orders of magnitude) performance improvement during model scoring. Changing your data repository into an “analytical database,” to be a bit trite, changes everything.

Ajay- Oracle has a huge India research presence-how do you think it can help popularize data mining driven decision making in developing countries.

Charlie- There are a lot of smart people across this planet. Today, with every Oracle Database having the capability to analyze data in place, the possibilities for clever people to create new applications, new use cases, and services are endless. I am always amazed that my bank’s ATM asks me what language I want to use every time I insert by card. Instead, I expect it to say, “Hello Mr. Berger, it is Friday night at 7 pm, did you want to take $150 (again) to go out on the town?” Or on Monday morning at 5:15 am, I would expect it to say, “Would you like to take out $250?” (Because the ATM application has mined your data and “guesses” that you are on your way to a flight for a trip and need to take some cash). These and many other new use cases require data, domain knowledge, and IT and analytical skills. I recommend that anyone interesting in new opportunities to read the Oracle Database (and Oracle Data Mining option) documentation and start thinking about creative new ways to exploit these new in-database analytical capabilities. The possibilities are endless. If you’ve watched the blockbuster movie Minority Report where Tom Cruise plays the Washington, DC pre-crime chief, John Anderton, who supervises the investigators who rely on 3 scientifically engineered beings who can see murders before they happen, you know what I mean.

Ajay- Could you describe some of your work at BIWA and value proposition for people to join it?

Charlie- Sure. BIWA is a community of like mined professional who use Oracle data warehousing, business intelligence and analytical (BIWA) features of Oracle’s database-centric technology. The mission of BIWA (www.oraclebiwa.org) is to share “best practices” and novel and interesting use cases of Oracle’s BIWA technology. As BIWA is free to join and our mission is to share technical knowledge, we’ve grown to thousands of members. We’ve held two BIWA Summits and are now organizing our third BIWA Summit, which will be held next year in April as part of Collaborate 2010 at the Mandalay Bay Hotel in Las Vegas. We’ve started a BIWA Wednesday TechCast Series that is open to anyone to present who is willing to publicly share an interesting use case or his or her knowledge and expertise.

Ajay- For anyone interested in Oracle Data Mining, how would you recommend them to quickly get started?

Charlie- I encourage all prospective Oracle Data Mining users to review the content at the Oracle Data Mining site on the Oracle Technology Network (OTN). Aside from it coming up as the first result in any Google Search for “Oracle Data Mining”, they can reach the web site here:

http://www.oracle.com/technology/products/bi/odm/index.html Highlights include:

1. Oracle Data Mining 11g Release 2 overview presentation
2. Oracle Data Mining 11g Release 2 white paper
3. Oracle Data Mining 11g Release 2 data sheet
4. Algorithm technical summary with links to Documentation
5. Recording of a recent BIWA TechCast with full presentation and several demos.
   • July 22, 2009, Oracle Data Mining Overview and Demos
     by Charlie Berger (slides, recording 37MB)
6. Getting Started with Oracle Data Mining OTN page with instructions to download the optional and free Oracle Data Miner GUI, Step-by-Step Tutorial and demo datasets
7. Oracle Data Mining Discussion Forum on OTN (great for posting questions and getting answers)
8. Sample Code (examples of ODM SQL and Java APIs in several use cases; great for developers)

I hope this helps people get going putting the data they have to beneficial use. Good luck to all!

Ajay- What does Charlie Berger do when not creating complex math based algorithms?
Charlie- I love to fish and caught a nice sized fish yesterday. I love to play tennis and Frisbee. In the winter, I ski whenever I get a chance. Like most, I love to travel and see new places. I’ve had the opportunity to travel to a number of interesting places either for work or with my family for vacation. This summer, I was at the Carolina beaches boogie boarding in the waves and chillaxing reading a good book. I’m currently simultaneously reading Outliers by Malcolm Gladwell, Predictably Irrational, by Dan Ariely, and Cryptonomicon, by Neal Stephenson and Neuromancer, by William Gibson.

Biography-

Charles R. Berger is senior director of product management, life sciences and data mining, at Oracle Corporation. You can follow his words here at http://twitter.com/CharlieDataMine

Ajay- Explain briefly what it is you do in the business intelligence space.
Terri- I am a freelance writer creating marketing material for BI vendors. I create case studies, white papers, brochures, articles, web content and short copy like e-mails and postcards. Because I’m still a techie girl at heart, I also design and develop websites including Wordpress customization which is quite popular now.

Ajay- How did you come to specialize in marketing for BI?
Terri- In the late ‘90s I went to work as a web developer for a large telecommunication company. In just a few years, I began managing the development group. Then I was asked to come to a young wireless company and manage their reporting group. After building out a solid enterprise reporting system (that still stands today), I went after a more holistic approach to reporting and analysis and created one of the first BICCs (business intelligence competency centers) in the country. Our group managed the oversight of the entire BI program including strategy, training, data quality, end user support, and BI communications. After some shifting in the winds, I knew it was time to move on.
In looking for that next “thing” I did a fair amount of soul searching. I knew I wanted a career that was both flexible and portable. I thought, “Why not use my experience in BI to create the very things I had consumed as a customer?” That’s when Advanced Marketing Collateral was born.

Ajay- How do you see the BI market place changing?
Terri- I guess that’s part of why I’ve been drawn to the BI field all these years. It just continues to change and improve. Just when you think they’ve done it all, a totally new concept emerges. However, I don’t envy the vendors. The competition is always at your heels. It seems like all the vendors have a solution for every industry or business line (though not so much for the emerging corporate sustainability area). Vendors have to continually experiment with new features and directions. Some will succeed and others will fail. It’s increasingly important and now even easier to communicate with both potential and existing customers, letting them know where you’re going and why, getting feedback, and just getting them to know more about the personality of your company. This can create such a strong bond.

Ajay- So how has that changed the way vendors should market their BI products?
Terri- It used to be you took out ads in the various BI publications, published and sent 12 page white papers, put up a website with technical descriptions for your products, and sent your sales force off to do “dog-and-pony” shows at prospective customer sites. Most of that is still valid, but we have so many more channels now. Combine that with the increased speed of the business and decreased attention span and you suddenly see that what used to work is not as effective anymore. Prospects want relevant information that they can quickly absorb and use to make decisions. This means more small bites which is why Twitter, blogging, and even Facebook are increasing in use in the B2B and especially BI space.

Ajay-You mentioned corporate sustainability. What is that and how does it relate to BI?
Terri- Corporate sustainability, also known as corporate social responsibility (CSR) is looking beyond just the financial returns of a company and also includes both social and environmental returns. Many of the Fortune 500 companies have already implemented CSR programs and some even have a dedicated C-level person in charge. With corporate strategies now coordinated three ways (financial, social, environmental), it’s even more important that metrics are tracked to ensure positive progress and alert where issues may be rising. Because CSR is so data-dependent, it makes good sense that BI vendors should be supporting this emerging area. I have only seen two major vendors play in this space (all of CSR, not just green data centers), so it has a lot of potential.

Ajay- Describe what you do for work life balance.
Terri- I am thoroughly enjoying what I do. I’m passionate about BI and its value to the business. I have many great relationships with many wonderful people in the BI space, and I have that portable, flexible career I’ve been looking for. When I’m not working on BI marketing projects, you’ll likely find me volunteering at our local animal shelter, walking and feeding dogs. I have a big heart for animals, especially homeless ones. I’ve also become very interested in photography and have had fun dabbling in that.

Biography
Terri Rylander is a freelance writer, who after 10 years as a customer, now produces the very content she used to consume. She works with large, Fortune 1000 companies and small startup vendors to build up content resources including case studies, white papers, brochures, articles, and other marketing collateral. She also does website development and Wordpress customization. You can find Terri on Twitter on @BIMarcom or visit her blog at blog.chooseamc.com or site at www.chooseamc.com.

Ajay: Describe your career in science and in business intelligence. How would you convince young students to take more maths and science courses for scientific careers.

Dylan: My main education for the profession was a degree in Information Technology and Software Development. No surprises what my first job entailed – software development for an IT company!

That role took me straight into the trials and tribulations of business intelligence and data quality. After a couple of years I went freelance and have pretty much worked for myself ever since. There has been a constant thread of data quality, business intelligence and data migration throughout my career which culminated in me setting up the more recent social media initiatives to try and pull professionals together in this space.

In all honesty, I’m probably the worst person to give career advice Ajay as I’m a hopeless dreamer. I’ve never really structured my career. I fell into data quality early on and it has led me to work in some wonderful places and with some great people, largely by accident and fate.

I have a simple philosophy, do what you love doing. I’m incredibly lucky to wake up every day with an absolute passion for what I do. In the past, whenever I have found myself working in a situation that I find soul destroying (and in our profession that can happen regularly) I move on to something new.

So, my advice for people starting out would be to first question what makes them happy in life. Don’t simply follow the herd. The internet has totally transformed the rules of the game in terms of finding an outlet for your skills so follow your heart, not conventional wisdom.

That said, I think there are some core skills that will always provide a springboard. Maths is obviously one of those skills that can open many doors but I would also advise people to learn about marketing, sales and other business fundamentals. From a business intelligence perspective it really adds an attractive dimension to your skills if you can link technical ability with a deeper understanding of how businesses operate.

Ajay You are a top expert and publisher on BI topics. Tell us something about

a) http://www.datamigrationpro.com/

b) http://www.dataqualitypro.com/

c) Involvement with the DataFlux community of experts

d) Your latest venture http://www.dqvote.com

Dylan- Data Migration Pro was my first foray into the social media space. I realised that very few people were talking about the challenges and techniques of data migration. On average, large organisations implement around 4 migration projects a year and most end in failure. A lot of this is due to a lack of awareness. Having worked for so long in this space I felt it was time to create a social media site to bring the wider community together. So we now have forums, regular articles, tools and techniques on the site with about 1400 members worldwide plus lots of plans in the pipeline for 2010.

Data Quality Pro followed on from the success of Data Migration Pro and our speed of growth really demonstrates how important data quality is right now. Again, awareness of the basic techniques and best-practices is key. I think many organisations are really starting to recognise the importance of better data quality management practices so a lot of our focus is on giving people practical advice and tools to get started. We are a community publishing platform, I do write regularly but we’ve always had a significant community contribution from expert practitioners and authors.

I didn’t just want to take a corporate viewpoint with these communities. As a result they are very much focused on the individual. That is why we post so many features on how to promote your skills, search for work, gain personal skills and generally get ahead in the profession. Data Quality Pro has just under 2,000 members and about 6,000 regular visitors a month so it demonstrates just how many people are really committed to learning about this discipline as it impacts practically every part of the business. I also think it is an excellent career choice as so many projects are dependent on good quality data there will always be demand.

The DataFlux community of experts is a great resource that I’ve actually admired for some time. I am a big fan of Jill Dyche who used to write on the community and of course there is a great line-up on there now with experts like David Loshin, Joyce Norris-Montanari and Mike Ferguson so I was delighted to be invited to participate. DataFlux have sponsored our sites from the very beginning and without their support we wouldn’t have grown to our current size. So although I’m vendor independent, it’s great to be sharing my thoughts and ideas with people who visit their site.

DQVote.com is a relatively new initiative. I noticed that there was some great data quality content being linked through platforms like Twitter but it would essentially become hard to find after several days. Also, there was no way for the community to vote on what content they found especially useful. DQVote.com allows people to promote their own content but also to vote and share other useful data quality articles, blogs, presentations, videos, tutorials – anything that adds value to the data quality community. It is also a great springboard for emerging data quality bloggers and publishers of useful content.

Ajay- Do you think BI projects can be more successful if we reward data entry people, or at least pay more for better quality data rather than ask them to fill in database tables as fast as they can? Especially in offshore call centres.

Dylan- Data entry is a pet frustration of mine. I regularly visit companies who are investing hundreds of thousands of pounds in data quality technology and consultants but nothing in grass-roots education and cultural change. They would rather create cleansing factories than resolve the issues at source.

So, yes I completely agree, the reward system has to change. I personally suffer from this all the time – call centre staff record incorrect or incomplete information about my service or account and it leads to billing errors, service problems, annoyance and eventually lost business. Call centre staff are not to blame, they are simply rewarded on the volume of customer service calls they can make, they are not encouraged to enter good quality data. The fault ultimately lies with the corporations that use these services and I don’t think offshore or onshore makes a difference. I’ve witnessed terrible data quality in-house also. The key is to have service level agreements on what quality of data is acceptable. I also think a reward structure as opposed to a penalty structure can be a much more progressive way of improving the quality of call-centre data.

Ajay- What are the top 5 things that you can help summarize your views on Business Intelligence – assume you are speaking to a class of freshmen statisticians.

Dylan- Business intelligence is wholly dependent on data quality. Accessibility, timeliness, accuracy, completeness, duplication – data quality dimensions like these can dramatically change the value of business intelligence to the organisation. Take nothing for granted with data, assume nothing. I have never, ever, assessed a dataset in a large business that did not have some serious data defects that were impacting decision making.

As statisticians, they therefore possess the tools to help organisations discover and measure these defects. They can find ways to continuously improve and ensure that future decisions are based on reliable data.

I would also add that business intelligence is not just about technology, it is about interpreting data to determine trends that will enable a company to improve their competitive advantage. Statistics are important but freshmen must also understand how organisations really create value for their customers.

My advice is to therefore step away from the tools and learn how the business operates on the ground. Really listen to workers and customers as they can bring the data to life. You will be able to create far more accurate dashboards and reports of where the issues and opportunities lie within a business if you immerse yourself with the people who create the data and the senior management who depend on the quality of your business intelligence platforms.

Ajay- Which software have you personally coded or implemented. Which one did you like the best and why?

Dylan- I’ve used most of the BI and DQ tools out there, all have strengths and weaknesses so it is very subjective. I have my favourites but I try to remain vendor neutral so I’ll have to gracefully decline on this one Ajay!

However, I did build a data profiling and data quality assessment tool several years ago. To be honest, that is the tool I like best because it had a range of features I still haven’t seen implemented so far in any other tools. If I ever get chance, and if no other vendor comes up with the same concept, I may yet take it to market. For now though, two young kids, two communities and a 12 hour day mean it is something of pipedream.

Ajay-What does Dylan Jones do when not helping data quality of the world go better.

Dylan- I’ve recently had another baby boy so kids take up most of whatever free time I have left. When we do get a break though I like to head to my home town and just hang out on the beach or go up into the mountains. I love travelling and as I effectively work completely online now, we’re really trying to figure out a way of combining travel and work.

Biography-

Dylan Jones is the founder and editor of Data Quality Pro and Data Migration Pro, the leading online expert community resources. Since the early nineties he has been helping large organisations tackle major information management challenges. He now devotes his time to fostering greater awareness, community and education in the fields of data quality and data migration via the use of social media channels. Dylan can be contacted via his profile page at http://www.dataqualitypro.com/data-quality-dylan-jones/ or at http://www.twitter.com/dataqualitypro

Now you can import your Gmail attachments Google docs into slideshare, for much better professional sharing within your office.

Here is an embedded SlideShare ppt called Google Hacks, note the much better visual appeal in this vis a vis your Google Docs.

Well as for the Stanford dropouts this is what happens when you dont complete your Phd education.

Citation- http://www.slideshare.net/rickdog/google-hacks

As per Cloud Computing and Office productivity goes,

Harvard Dropouts (Microsoft) 1- Stanford Dropouts ( Google) 0

Unless Google creates a cloud version of Open Office- but who needs that anyway?

who needs search- just ctrl F

I am creating a new website www.dudeofdata.com which will be more irreverant, more sarcastic on corporate hypocricy in analytics and statistics. It will feature the hack kings of mashing software and would be expunged for lots of stuff. Strategic intention would be to make statistics “cool’ for my fellow hosts in America- I see less American faces in science blocks than East Asian faces.

You can follow the twitter page www.twitter.com/dudeofdata in the meantime for the launch. It should take me less than an hour to set up the Wordpress instal and CMS- but my hours are spent learning classes, homework, washing dishes and cooking food, ( and also learning rock climbing and playing soccer with the people).

In the meantime here is a salute to Airlines- I recently flew 30 hours from India to USA and can relate.

Ajay- Describe your career in science. How do you think science careers can be made more popular to young students.

Sarah- Other than a little time in electronics engineering, I have spent all my career in the computer industry. Science degrees give you the kind of training and credentials that you need for a good start in the world of work. There are many jobs in the applied sciences domain e.g. electronics and computer science, but science graduates get into a number of other fields e.g. the financial markets. We are having a crisis in science education here in the UK where a number of good universities have had to close some of their science departments. This is primarily due to the lack of students. The trouble is that science is not considered to be “cool” amongst the would be students. I believe the way to tackle this is by making science more interesting at school to inspire young people to take it up in tertiary education. Without science and innovation we will be on a slippery slope to economic decline. TV and the media in general do not help with their constant portrayal of scientists as geeks. Perhaps we should find a couple of good looking scientists to promote in the media to compete for the attention of young people against celebrities. Do you know any Brad Pitt look alike scientists?

Ajay- I feel the Business Intelligence world is overwhelming male in terms of statistics. Do you agree- what makes BI and data mining a not so attractive career traditionally for women.

Sarah- I agree. I think it reflects the general trends in the take up of science and mathematics by girls at university and also the number of women in management positions. Whatever we do to increase the number of women in those areas is likely to have an effect on the number of women in BI. My view is that we need to make subjects such as statistics more appealing and relevant to girls at school. I believe we can do that by teaching analysis of trends in some weird and wonderful topics with some funny facts thrown in to make it fun. I also think that women tend to be less confident than men in trying technical or scientific subjects. Again, I do not think that the culture of celebrity helps. It puts pressure on girls for the wrong reasons and discourages them from pusuing some worthwhile careers.

Ajay- What are your views on Government spending to be measured by Business Intelligence tools the same way as corporate spending is measured by it. Do you think that some of the stimulus package to shore up failing banks was big enough to educate every high school graduate through college ( in both UK and the US)

Sarah- I am a proponent of BI in public sector and have written articles and blogged about it. There are many examples of Government bodies getting into financial or capacity difficulties due to the lack of good data and actionable information. BI can be applied at departmental level to improve departmental outcomes and services. BI at multi-agency level can help make the customer/citizen’s journey a positive experience through the maze of public services and help with efficiency targets. The trouble is bringing data together from multiple-agencies within legal frameworks such as the Data Protection Act in the UK and also without losing the trust of voters that the data is not being shared amongst Government bodies for some sinister reason.

Ajay- If you can not measure it, you cannot manage it. Do you think measuring carbon footprint of organizations is the first step to managing environmental fallout. How can IT help make the world greener?

Sarah- Yes, it is a start but like any other performance measure, it has to be linked to strategic objectives and the findings have to be acted upon in order to reduce the environmental fall out and towards achieving objectives. IT can play a number of different parts in this: automate the necessary processes, capture data and then enable the measuring, monitoring and reporting part of the initiative. IT of course has to put its own house in order to become a sustainable service that helps with the overall green objectives.

Ajay- Increasing number of personal data is now available on the web about consumers, just as financial records are available from credit bureaus. How long do you think will it take for software to catch up in text mining for propensity to buy, or risk behaviour by adding this social media data to traditional data sources.

Sarah- I do not believe it will take long at all. Text analytics solutions can be trained and put to analyse the sort of information that you are interested in. Of course where personal information is concerned legal requirements have to be complied with.

Ajay- What are your views on social network analysis and how it can help BI measurement and predictive analytics.

Sarah- It is an interesting and developing area that can augment BI. From an analysis point of view, I think it is important to validate the source and the accuracy of information and to give it some kind of relevance and quality mark. The information must then be treated in accordance to its mark so that pointless hype can be eliminated. Although, in some cases hype might matter e.g. we need to understand if it is likely to affect our business. For that, we need to predict when hype or word of mouth can rapidly spread through a social network. The network itself can provide some information revealing its influencer/follower relationships so that hype can be predicted by what the influencers are saying.

Ajay- What are the biggest, most common mistakes you see in implementation of Information technology strategy.

Sarah- My answer to that question could take several pages and I speak from bitter experience. For now lets just mention a few examples: Trying to do things too fast, not gaining end-user buy-in or input, lack of communication, failing to appreciate the extent of skills requirements and supposedly cutting costs by avoiding consultants, not enough hand-holding of end-users on roll out and so on.

Ajay- What do you do while not working or writing to relax. How important is the work-life balance for analysts.

Sarah- Pilates keeps me sane during the week. Other times I enjoy playing sports and doing leisure activities with my family and friends. Work/life balance is very important to me – so much so that I took a break from corporate life during a peak in my career to raise a family and have never regretted it. Many years ago I read a poem by Nadine Stair called “Afterwords: If I could live it over….”. The best line says “If I had to live my life over again I would eat more ice cream”. Joking apart, I do not want to be old and regretting all the things that I have not done.

Sarah Burnett is a Senior Analyst with Ovum. An experienced analyst and consultant, Sarah has worked in a variety of IT roles over the last twenty years including software development, programme management and project management. She provides analysis and thought leadership on Business Intelligence software and market; she also provides expert analysis of public sector IT developments and trends. Sarah is a regular contributor to the company’s monthly journals providing articles as well as writing a monthly column on public sector IT. She is a regular speaker at conferences and provides personalised advice and consultancy to Ovum clients.

Biography-

Since joining the Ovum-Butler organisation three years ago, Sarah has co-authored a number of in-depth reports and recently led the research and production of the Business Intelligence (Corporate Performance Management) report.

Sarah holds a BSc in Physics and Electronics as well as an MSc in Applied Optics. She has Prince2 practitioner qualifications and is a member of the British Computer Society. Sarah can be found on-line on Twitter -sarahburnett- and on her own blog, Sarah Burnett’s Web Musings.

Ajay- Describe your career journey. What interested you in science? How do you think we can help more young people get interested in science?

Eric- I was a classic underachiever in school. I was bright, but was generally disinterested in academics, and focused on… well, other things at the time. However, I had always excelled in math and science, and actually paid attention those classes.

I was a high school junior when my school acquired its first computers: Apple IIs. There were no formal computer courses, so instead of study hall, I would go to the lab and tinker. Sure, I would join a few other geeks (well before it was cool to be such) for a few primitive games, but would spend the majority of my time reading about the Basic programming language and coding graphic designs, math formulas and simple games.

I loved it so much that I had decided to pursue computer science as a college major before my senior year and it went into my yearbook entry. Fortunately, my relatively high SAT scores offset my poor high school GPA and squeaked me into the University of Pittsburgh’s trial-by-fire summer program. It was the first time I really felt I had to perform (or else) and had to work hard to overcome poor study habits — but rose to the occasion with room to spare.

I’m glad I did not realize at the time that Pitt was #9 in the nation for computer science. I did have a hint though when I realized the extremely high attrition rate. In the end, our freshman class of 240 graduated 36. I did make it through the freshman year that trimmed the first half of the original group, but was a casualty my sophomore year when I fell short of a passing grade in a core CS course that was only offered annually. I repeated it the following year and graduated with extra credits – to include a directed study in table tennis (no kidding).

I loved the programming assignments but loathed the tests. After slogging through the program and graduating, I took a three month break. I figured it would be my last opportunity to be free of responsibility for that period of time possibly until retirement – and so far, I’m right.

Then, my cousin who graduated with me told me about a neural computing software tools company in Pittsburgh, called NeuralWare. I was always intrigued by “artificial intelligence”, but they were seeking a technical support representative. I realized my junior year that I did not want to code or remain on the technical side for a living, but go into business development, project management, business management and entrepreneurship. Yet, after having survived the majority of the attrition, I did want to complete my technical degree, then seek the business angle.

A short while later, NeuralWare contacted me again to start up their sales operation (a role previously fulfilled a co-founder). This was the start I was seeking: cut my teeth in business for highly technical products. I participated in numerous training sessions for neural computing and related technologies and loved it. The notion that the computer could leverage mathematics that emulated the basic learning function of the brain, or treat a formula like a gene – split it, mutate it, test and progress toward the most fitting solution was beyond exciting to me. So much so, that I’ve not left the technology in the 19 years since.

Drawing others to science, I believe is more a matter of nature over nurture. I am the father of twin boys who couldn’t have greater differences in interests, personalities and talents. In that spirit, I believe that science should be made readily available, involve both theory and practice, and be presented in a manner that motivates those who are drawn to science to excel. But I don’t believe science can be effectively pushed to those whose inherit interests and passion lie elsewhere (reference the character Neil Perry in The Dead Poet’s Society).

Ajay- Describe the path that The Modeling Agency has traveled. What is your vision for it for the future.

Eric- The Modeling Agency (TMA) was established as a highly structured formal network of senior-level consultants in January of 2000. TMA’s initial vision (and sustained slogan) was to “provide guidance and results to those who are data-rich, yet information-poor.” I still have not encountered an organization that holds a larger bench of senior-level data mining consultants and trainers. And to be senior-level, TMA consultants must be far more than technically steeped in data mining. TMA’s senior consulting staff are business consultants first – not rushing to analyze data, but assessing an organization’s environment and designing a fitting solution to resources that support stated objectives.

There are three primary divisions to TMA: training, consulting and solutions. Each division is part of an overarching business and technology maturation process. For example, training generates technology advocates for data mining that encourages consulting engagements which at times lead to productizable vertical market services that create solutions which allow other organizations to capitalize on the risk that pioneering organizations had undertaken, and springboard on the return realized by implementations within their vertical – which leads to new discoveries and innovations that feed back to training.

Beyond further developing the brand of TMA’s quickly emerging niche (described later), our future vision involves developing two specific types of vendor partnerships to allow TMA to redirect the substantial margins enjoyed by its clients through the application of predictive modeling into a residual stream of income to accelerate the growth of TMA itself. While this operation is confidential, we will be pleased to tell our future clients that we do indeed apply our services for the benefit of our own business.

Ajay- Describe the challenges and opportunities in modeling through recent innovations. i.e social network analysis software and increasing amounts of customer text data available on social media.

Eric- Please allow me to shift the focus of this question slightly. So many organizations are still making their way down the Business Intelligence chain to applying predictive modeling on standard operational data, that social network analysis and customer text analytics remains more of a research endeavor in my opinion. As a practical applications company, TMA focuses its experience in pragmatically applying its business problem solving creativity on operational and transactional data enriched by demographic and psychographic attributes. I feel that the areas of social media and social network analysis are not yet mature enough to be formalized as established practice on TMA’s menu of service offerings.

Having said that, the greatest challenges in predictive modeling are no longer in applying the methodological tactics, but rather in the comprehensive assessment, strategic problem design, project definition, results interpretation and ROI calculation. Popular data mining software is now highly effective at automating the tactical model building process – many packages running numerous methods in parallel and selecting the best performer.

So, the challenges that remain today are in tackling the tails of the process as mentioned above. This is where TMA’s expertise is focused and where our niche is quickly emerging: guiding organizations to establish their own internal predictive analytics operation.

Ajay- In the increasing game of consolidation of business intelligence vendors and data mining and analytics, which are the vendors that you have worked with and what are their relative merits.

Eric- TMA has established formal partnerships with several popular data mining tool vendors and services companies. Despite these alliances, TMA remains vendor neutral and method agnostic for clients that approach TMA directly. Having said that, I will make a general statement that there is notable merit for the organizations that recognize that they must ensure their client’s success in the full implementation cycle of data mining – not just provide a great tool that addresses the center.

In fact, it was one of TMA’s earliest partners who saw the value in teaming with TMA to support the ends of the data mining process (assessment, business understanding project definition and design, results interpretation, implementation) while their solution addressed the middle (data preparation and modeling). They recognized that as great as their tool was, it was still hitting the shelf soon after the sale. The realized that their clients were building very good models that answered the wrong questions, or were uninterpretable and incapable of implementation.

TMA soon recognized that these excellent tools combined with TMA’s strategic data mining mentorship and counsel provided the capability for organizations to essentially establish their own internal predictive analytic practice with existing business practitioners – not requiring senior statisticians or PhDs. This has become a popular and fast growing service, for which TMA’s large bench of senior-level data mining consultants is perfectly suited to fulfill.

And the best candidates for this service are those organizations who have attempted pilots or projects but fell short of their objectives. And while the acquisition of SPSS (who licenses a reputable predictive analytics tool, “PASW”) by IBM (the gold standard for IT and BI services and solutions) may be the closest competition that TMA may encounter, TMA enjoys a substantial head start and foothold with its numerous formal alliances, vendor neutrality and sizable client list specific to predictive modeling. TMA is quickly becoming the standard to turn to for progressive organizations that realize internalizing predictive analytics is not just a matter of when rather than whether, but that it is within their grasp with TMA’s guidance and the right tool(s).

Ajay- What do people at The Modeling Agency do for fun?

Eric- Our interests are as diverse as we are geographically disbursed. One of our senior consultants is a talented and fairly established tango dancer. He’s always willing to travel for assignments, as he’s anxious to tap into that city’s tango circuit. Another consultant is an avid runner, entering marathons and charity races. One common thread that most of us share is our dedication to parenting. We all love trips and time with our children. In fact, I’m writing this on a return trip from Disney World on the Auto Train with my 5 year old twin boys – a trip I know I’ll recall fondly through my remaining years.

Bio

Eric A. King is President and Founder of The Modeling Agency (TMA), a US-based company started in January 2000 that provides training, consulting, solutions and a popular introductory webinar in predictive modeling “for those who are data-rich, yet information-poor.”  King holds a BS in computer science from the University of Pittsburgh and has over 19 years of experience specifically in data mining, business development and project management.  Prior to TMA, King worked for NeuralWare, a neural network tools company, and American Heuristics Corporation, an artificial intelligence consulting firm.  He may be reached at eric@the-modeling-agency.com or (281) 667-4200 x210.

1) Karl Rexer – Rexer Analytics

http://www.decisionstats.com/2009/06/09/interview-karl-rexer-rexer-analytics/

Karl produces one of the most respected surveys that captures emerging trends in data mining and technology. Karl was also one of the most enthusiastic people I have interviewed- and I am thankful for his help in getting me some more interviews.

2) Gregory Piatesky Shapiro

One of the earliest and easily the best Knowledge Discoverer of all times, Gregory produces www.kdnuggets.com and the newsletter is easily the must newsletter to be on. Gregory was doing data mining , while the Google boys were still debating whether to drop out of Stanford or not.

http://www.decisionstats.com/2009/08/13/interview-gregory-piatetsky-kdnuggets-com/

3) SPSS – Oliver Jouve

This was a special interview coming after the rebranding of SPSS products to PASW. SPSS products are good- I like the clean intutive and neat GUI best of all.

http://www.decisionstats.com/2009/04/30/interview-spss-olivier-jouve/

4) KXEN – Francoise Soulie Fogelman

http://www.decisionstats.com/2009/03/27/interview-franoise-soulie-fogelman-kxen/

I briefly consulted for KXEN advising on social media, but this interview preceded that contract. Francoise is such knowledge and easily the KXEN software automation for building models is one must try software. I am also apprectiative of Roger Hadaad, who was the first company chairman  to give an interview to me, back in late 2008, when Decisionstats was still finding it’s way. Personal favourites apart- KXEN software impressed me with the speed of creating a model and portability across various platforms. This interview is a great revealer for that.

5) SPSS Jon Peck

http://www.decisionstats.com/2009/03/17/interview-jon-peck-spss/

Jon Peck was and remains SPSS ’s top technical guy and well respcted within the SPSS community. Wherever Jon goes next, he would take a train of admirers and clients. Here he explains the R and SPSS link along with the Python functionality supported by SPSS.

Business Intelligence

1) Jill Dyche

http://www.decisionstats.com/2009/06/30/interview-jill-dyche-baseline-consulting/

Jill is a fabulously wise and experienced person with a great writing style. Here answers were some of the most educative I have seen in Bi writing.

2) Peter Thomas

http://www.decisionstats.com/2009/07/02/peter-james-thomas-bi/

The best of British BI is epitomized by Peter Thomas, and he is truly a European giant when it comes to the field. His worst weakness is a tendency to disappear when Test cricket is around- but that is

eminently understable. I can relate to the cricket as well.

3) Karen Lopez

http://www.decisionstats.com/2009/07/28/interview-karen-lopez/

Karen gives an excellent insight on creating mock ups or data models before actual implementation. She has worked on it for three decades and her wisdom is clearly visible here.

Data Quality

Data quality is such an overlooked and easy to fix issue, that I belive any BI vendor that builds the best, most robust data quality architechture will gain the maximum Pareto like benefits out of results. Curiously competing BI vendors will often compete on price, grahics appeal, etc etc, but the easy Garbage In Garbage Out rule is something they should consider. The Data Quality Interviews gave me an important tutorial in these aspects of data management.

1) Jim Harris

http://www.decisionstats.com/tag/jim-harris/

Jim is an one man army when it comes to evangelizing data quality and his OCDQ blog is widely read and cited.

2) Steve Sarsfield

http://www.decisionstats.com/2009/08/13/interview-steve-sarsfield-author-the-data-governance-imperative/

His excellent book is the one must read item that people in cost cutting corporations should buy especially if they are considering to go down the Davenport competing on analytics model.

( To be continued- Part 3 Modeling and Text Mining

Part 4 Social Media

Part 5 Humour and Poetry )

The site has now gotten more than 50,000 views since being launched in Dec 2007.

Thank you everyone for your help in this. More importantly the quality of comments has been fabulous. Since I am out of ideas for the rest of the week- here is a best of posts collection.
Here are some of the most favorite articles as measured by number of page views. I have personal fovurites as well, but these are just the ranks as per page views and how they measure up.

Top 5 Interviews

1) Interviews with SAS Institute leaders- I have found generally great professionalism from SAS Institute people. This is surprising because comin from an open source background, SAS is often looked as a big brother. I find that more of a perception and less of a reality as the company continues to innovate.

a) with John Sall, founder SAS Institute- This is really the biggest interview I did in terms of the person involved. To my surprise ( I wasnt expecting John to say yes) the interview was really frank, and it came very fast. The answers seem to be written by John himself.

Quote- Quantitative fields can be fairly resistant to recession- John Sall.

http://www.decisionstats.com/2009/07/28/interview-john-sall-jmp/

b) Interview with Anne Milley, Director, Product Marketing , SAS Institute- This is a favourite because it came very soon after the NYTimes article on R etc. One of my personal opinions is that the difference between great and good leaders is often the fact that great leaders are humble enough  to learn and then build on their strengths. It ran in two parts- and I was really appreciative of the in-depth answers that Anne wrote.

Quotes-

Analytics continues to be our middle name.

Customers vote with the cheque book.

http://www.decisionstats.com/2009/03/06/interview-with-anne-milley-sas-ii/

http://www.decisionstats.com/2009/03/04/interview-anne-milley-sas-part-1/

c) Jim Davis- CMO, SAS Institute

This came after Jim Davis issued his views on Business Intelligence as an over used term, contrasting it with business analytics.and British BI expert Peter Thomas called him out on that.  As a data mining student I wanted to side with Jim Davis, but as a business intelligence learner I found Peter’s views great to read as well. I am still trying on finding this the difference between BI and BA. We need more of business analytics and better quality business intelligence to prevent disasters like mortgage forecasting etc

Traditional business intelligence (BI) as we know it is outdated and insufficient.

http://www.decisionstats.com/tag/jim-davis/

d) Interview with Alison Bolen -Common sense would dictate that smaller companies would be better adapters of social media, given the low cost and high visibilty. Yet in all aspects that I study, from search engine optimization to twitter penetration, Facebook pages number of fans, blogs -the maximum quantity and quality of content continues to come from the communications team at the biggest and oldest business analytics vendor, SAS. Alison Bolen is one of the best social media and communications editor I have seen in the BI industry.

Quote- We’re so used to writing white papers, brochures, maginze articles that the concept of writing 200 words on a topic from your day is very foreign to us.

http://www.decisionstats.com/tag/alison-bolen/

e) Interview wit Gary Cokins- I am still trying to read his book, but the fall semester started and I am very impressed by Gary Cokins lucid and clear views on using and integrating business analytics for strategic change and improvement in performance across the organization.

http://www.decisionstats.com/2009/06/18/interview-gary-cokins-sas-institute/

2) Interviews with R professionals.

a) David Smith- Director of Community at Revolution Computing.

http://www.decisionstats.com/2009/05/29/interview-david-smith-revolution-computing/

b) Richard Schultz, CEO Revolution Computing

http://www.decisionstats.com/2009/01/31/interviewrichard-schultz-ceo-revolution-computing/

REvolution continues to be a very very good tech company to watch for. Their product enhancements are great, particulraly their work in enhancing R for speed, and their community management is superb. David’s excellent blog is almost like reading a journal in itself on R and a must read for any R learner.

c) Interview with Paul van Eikeren, Inference for R

http://www.decisionstats.com/2009/06/04/inference-for-r/

Inference for R is a good example for open source and properietary systems working together with both GPL compliance and best of all- customer ease in using software.

d) Interview with Dr Graham Williams

People come up to me and say R has a diificult syntax. It has a very very flexible syntax, which makes it powerful for usage. If all you wanted to do was make great data mining models, the software by Dr Williams is a pioneer in R GUIs.

Rattle continues to be a must install package in R, and this was a nice interview in the both the depth of experience as well breadth of perspective by the creator of Rattle, which is also having a commerical derivative now.

http://www.decisionstats.com/2009/01/13/interview-dr-graham-williams/

e) Interview with Bob Muenchen- This interview with the author of R for SAS and SPSS users was the first big interview I did, the book review was read many many times. In fact I now work in same university as Bob. Great common sensical views from an experienced statistical author and consultant.

http://www.decisionstats.com/2009/01/21/r-for-sas-and-spss-users/

( to be continued- this is a partial list)

Ajay- Please describe your career in science, many challenges and rewards that came with it. Name any scientific research, degrees teaching etc.

Gregory- I was born in Moscow, Russia and went to a top math high-school in Moscow. A unique  challenge for me was that my father was one of leading mathematicians in Soviet Union.  While I liked math (and still do), I quickly realized while still in high school that  I will never be as good as my father, and math career was not for me.

Fortunately, I discovered computers and really liked the process of programming and solving applied problems.  At that time (late 1970s) computers were not very popular and it was not clear that one can make a career in computers.  However I was very lucky that I was able to pursue what I liked and find demand for my skills.

I got my MS in 1979 and PhD in 1984 in Computer Science from New York University.
I was interested in AI (perhaps thanks to a lot of science fiction I read as a kid), but found a job in databases, so I was looking for ways to combine them.

In 1984 I joined GTE Labs where I worked on research in databases and AI, and in 1989 started the first project on Knowledge Discovery in data. To help convince my management that there will be a demand for this thing
called “data mining” (GTE management did not see much future for it), I also organized a AAAI workshop on the topic.

I thought “data mining” is not sexy enough name, and so I called it “Knowledge Discovery in Data”, or KDD.  Since 1989, I was working on KDD and data mining in all aspects – more on my page www.kdnuggets.com/gps.html

Ajay-  How would you encourage a young science entrepreneur in this recession.

Gregory- Many great companies were started or grew in a recession, e.g.
http://www.insidecrm.com/features/businesses-started-slump-111108/

Recession may be compared to a brush fire which removes dead wood and allows new trees to grow.

Ajay- What prompted you to set up KD Nuggets? Any reasons for the name (kNowledge Discovery Nuggets). Describe some key milestones in this iconic website for data mining people.

Gregory- After a third KDD workshop in 1993 I started a newsletter to connect about 50 people who attended the workshop and possibly others who were interested in data mining and KDD.  The idea was that it will have short items or “nuggets” of information. Also, at that time a popular metaphor for data miner was gold miners who were looking for gold “nuggets”.  So, I wanted a newsletter with “nuggets” – short, valuable items about Knowledge Discovery.  Thus, the name KDnuggets.

In 1994 I created a website on data mining at GTE and in 1997, after I left  GTE , I moved it to the current domain name www.kdnuggets.com .

In 1999, I was working for startup which provided data mining services to financial industry.  However, because of Y2K issues, all banks etc froze their systems in the second half of 1999, and we had very little work (and our salaries were reduced as well).  I decided that I will try to get some ads and was able to get companies like SPSS and Megaputer to advertise.

Since 2001, I am an independent consultant and KDnuggets is only part of what I am doing.  I also do data mining consulting, and actively participate in SIGKDD (Director 1998-2005, Chair 2005-2009).

Some people think that KDnuggets is a large company, with publisher, webmaster, editor, ad salesperson, billing dept, etc.  KDnuggets indeed has all this functions, but it is all me and my two cats.

Ajay- I am impressed by the fact KD nuggets is almost a dictionary or encyclopedia for data mining. But apart from advertising you have not been totally commercial- many features of your newsletter remain ad free – you still maintain a minimalistic look and do not take sponsership aligned with one big vendor. What is your vision for KD Nuggets for the years to come to keep it truly independent.

Gregory- My vision for KDnuggets is to be a comprehensive resource for data mining community, and I really enjoyed maintaining such resource for the first 7-8 years completely non-commercially. However, when I became self -employed, I could not do KDnuggets without any income, so I selectively introduced ads, and only those which are relevant to data mining.

I like to think of KDnuggets as a Craiglist for data mining community.

I certainly realize the importance of social media and Web 2.0 (and interested people can follow my tweets at tweeter.com/kdnuggets)  and plan to add more social features to KDnuggets.

Still, just like Wikipedia and Facebook do not make New York Times obsolete, I think there is room and need for an edited website, especially for such a nerdy and not very social group like data miners.

Ajay- What is the worst mistake/error in writing publishing that you did. What is the biggest triumph or high moment in the Nuggets history.

Gregory- My biggest mistake is probably in choosing the name kdnuggets – in retrospect,  I could have used a shorter and easier to spell domain name, but in 1997 I never expected that I will still be publishing www.KDnuggets.com 12 years later.

Ajay- Who are your favourite data mining students ( having known so many people). What qualities do you think set a data mining person apart from other sceinces.

Gregory- I was only an adjunct professor for a short time, so I did not really have data mining students, but I was privileged enough to know many current data mining leaders when they were students.  Among more recent students, I am very impressed with Jure Leskovec, who just finished his PhD and got the best KDD dissertation award.

Ajay- What does Gregory Piatetsky do for fun when he is not informing the world on analytics and knowledge discovery.

Gregory- I enjoy travelling with my family, and in the summer I like biking and windsurfing.
I also read a lot, and currently in the middle of reading Proust (which I periodically dilute by other, lighter books).

Ajay- What is your favourite reading blog and website ? Any India plans to visit.
Gregory- I visit many blogs on www.kdnuggets.com/websites/blogs.html

and I like especially
- Matthew Hurst blog: Data Mining: Text Mining, Visualization, and Social Media
- Occam’s Razor by Avinash Kaushik, examining web analytics.
- Juice Analytics, blogging about analytics and visualization
- Geeking with Greg, exploring the future of personalized information.

I also like your website decisionstats.com and plan to visit it more frequently

I visited many countries, but not yet India – waiting for the right occasion !

Biography-

(http://www.kdnuggets.com/gps.html)

Gregory Piatetsky-Shapiro, Ph.D. is the President of KDnuggets, which provides research and consulting services in the areas of data mining, web mining, and business analytics. Gregory is considered to be one of the founders of the data mining and knowledge discovery field.Gregory edited or co-edited many collections on data mining and knowledge discovery, including two best-selling books: Knowledge Discovery in Databases (AAAI/MIT Press, 1991) and Advances in Knowledge Discovery in Databases (AAAI/MIT Press, 1996), and has over 60 publications in the areas of data mining, artificial intelligence and database research.

Gregory is the founder of Knowledge Discovery in Database (KDD) conference series. He organized and chaired the first three Knowledge Discovery in Databases (KDD) workshops in 1989, 1991, and 1993. He then served as the Chair of KDD Steering committee and guided the conversion of KDD workshops into leading international conferences on data mining. He also was the General Chair of the KDD-98 conference.

Ajay-     Describe your career in science. How would you motivate young people today to take science careers rather than other careers
Dr Fayyad- My career started out in science and engineering. My original plan was to be in research and to become a university professor. Indeed, my first few jobs were strictly in basic Research. After doing summer internships at place like GM Research Labs and JPL, my first full-time position was at the NASA – Jet Propulsion Laboratory, California Institute of Technology.

I started in research in Artificial Intelligence for autonomous monitoring and control and in Machine Learning and data mining. The first major success was with Caltech Astronomers on using machine learning classification techniques to automatically recognize objects in a large sky survey (POSS-II – the 2nd Palomar Observatory Sky Survey).  The Survey consists of taking high resolution images of the entire northern sky. The images, when digitized, contain over 2 billion sky objects. The main problem is to recognize if an object is a star of galaxy. For “faint objects” – which constitute the majority of objects, this was an exceedingly hard problem that people wrestled with for 30 years. I was surprised how well the algorithms could do at solving it.

This was a real example of data sets where the dimensionality is so high that algorithms are better suited at solving it than humans – even well-trained astronomers. Our methods had over 94% accuracy on faint objects that no one could reliably classify before at better than 75% accuracy. This additional accuracy made all the difference in enabling all sort of new science, discoveries and theories about formation of large scale structure in the Universe.
The success of this work and its wide recognition in scientific and engineering communities let to the creation of a new group – I founded and managed the Machine Learning Systems group at JPL which went on to address hard problems in object recognition in scientific data – mostly from remote sensing instruments – like Magellan images of the planet Venus (we recognized and classified over a million small volcanoes on the planet in collaboration with geologists at Brown University) and Earth Observing System data, including Atmospherics and storm data.
At the time, Microsoft was interested in figuring out data mining applications in the corporate world and after a long recruiting cycle they got me to join the newly formed Microsoft Research as a Senior Researcher in late 1995. My work there focus on algorithms, database systems, and basic science issues in the newly formed field of Data Mining and Knowledge Discovery. We had just finished publishing a nice edited collection of chapters in a book that became very popular, and I had agreed to become the founding Editor-in-Chief of a brand new journal called: Data Mining and Knowledge Discovery. This journal today is the premier scientific journal in the field. My research work at Microsoft led to several applications – especially in databases. I founded the Data Mining & Exploration group at MSR and later a product group in SQL Server that built and shipped the first integrated data mining product in a large-scale commercial DBMS  – SQL Server 2000 (analysis Services). We created extensions to the SQL language (that we called DMX) and tried to make data mining mainstream. I really enjoyed the life of doing basic research as well as having a real product group that built and shipped components in a major DBMS.
That’s when I learned that the real challenging problems in the real-world where really not in data mining but in getting the data ready and available for analysis – Data Warehousing was a field littered with failures and data stores that were write-only (meaning data never came out!)  — I used to call these Data Tombs at the time and I likened them to the pyramids in Ancient Egypt: great engineering feats to build, but really just tombs.

In 2000 I decided to leave the world of Research at Microsoft to do my first venture-backed start-up company – digiMine. The company wanted to solve the problem of managing the data and performing data mining and analysis over data sets, and we targeted a model of hosted data warehouses and mining applications as an ASP – one of the first Software as a Service (SaaS) firms in that arena. This began my transition from the world of research and science to business and technology.  We focused on on-line data and web analytics since the data volumes their were about 10x the size of transactional databases and most companies did not know how to deal with all that data. The business grew fast and so did the company – reaching 120 employees in about 1 year.

After 3 years of doing high-growth start-up and raising some $50 million in venture capital for the company, I was beginning to feel the itch again to do technical work.
In June 2003, we had a chance to spin-off part of the business that was focused on difficult high-end data mining problems. This opportunity was exactly what I needed and we formed DMX Group as a spinoff company that had a solid business from its first day. At DMX Group I got to work on some of the hardest data mining problems in predicting sales of automobiles, churn of wireless users, financial scoring and credit risk analysis, and many related deep business Intelligence problems.

Our client list included many of the Fortune 500 companies. One of these clients was Yahoo!  — After 6 months of working with Yahoo! As a client they decided to acquire DMX Group and use the people to build a serious data team for Yahoo!  We negotiated a deal that got about half the employees into Yahoo! And we spun-off the rest of DMX Group to continue focusing on consulting work in data mining and BI.  I thus became the industry’s first Chief Data Officer. 

 The original plan was to spend 2 years or so to help Yahoo! Form the right data teams and build the data processing and targeting technology to deliver high value from its inventory of ads.
Yahoo! Proved to be a wonderful experience and I learned so much about the Internet. I also learned that even someone like me who worked on Internet data from the early days of MSN (in 1996) and who ran a web analytics firm still did not scratch the service on the depth of the area. I learned a lot about the Internet from Jerry Yaang (Yahoo! Co-founder) and much about advertising/media business from Dan Rosensweig (COO) and mTerry Semel (then CEO) and lots about technology management and strategic deal-making from Farzad (Zod) Nazem who was the CTO. As Executive VP at Yahoo!

I built one of the industry’s largest and best data teams and we were able to to process over 25 terabytes of data per year and power several hundred million Dollars of new revenue for Yahoo! Resulting from these data systems. A year after joining Yahoo! I was asked to form a new Research Lab to study much of what we did not understand about the Internet. This was yet another return of basic research into my life. I founded Yahoo! Research to invent the new sciences of the Internet, and I wanted them to be focused on only 4 areas (the idea of focus came from my exposure to Caltech and its philosophy in picking few areas of excellence). The goal was the become the best research lab in the world in these new focused areas. Surprisingly we did it within 2 years. I hired Prabhakar Raghavan to run Research and he did a phenomenal job in building out the Research organization. The four areas we chose were: Search and information navigation, Community Systems, Micro-economics of the Web, and Computational Advertising.  We were able to attract the top talent in the world to lead or work on these emerging areas. Yahoo! Research was a success in basic research but also in influencing product. The chief scientists for all the major areas of company products all came from Yahoo! Research and all owned the product development agenda and plans: Raghu Ramakrishnan (CS for Audience), Andrew Tomkins (CS for Search), Anrei Broder (CS for Monetization) and Preston McCaffee (CS for Marketplaces/Exchanges). I consider this an unprecendented achievement in the world of Research in general: excellence in basic research and huge impact on company products, all within 3-4 years.
I have recently left Yahoo! And started Open Insights (www.open-insights.com) to focus on data strategy and helping enterprises realize the value of data, develop the right data strategies, and create new business models. Sort of an ‘outsourced version” of my Chief Data Officer job at Yahoo!
Finally, on my advice to young people: it is not just about science careers, I would call it engineering careers. My advice to any young person in fact, whether they plan to become a business person, a medical doctor, and artist, a lawyer, or a scientist – basic training in engineering and abstract problem solving will be a huge assets. Some of the best lawyers, doctors, and even CEO’s started out with engineering training.
For those young people who want to become scientists, my advice is always look for real-world applications where the research can be conducted in their context. The reason for that is technical and sociological. From a technical perspective, the reality of an application and the fact that things have to work force a regiment of technical discipline and make sure that the new ideas are tested and challenged. Socially, working on a real application forces interactions with people who care about the problem and provides continuous feedback which is really crucial in guiding good work (even if scientists deny this, social pressure is a big factor) – it also ensures that your work will be relevant and will evolve in relevant directions. I always tell people who are seeking basic research: “some of the deepest fundamental science problems can often be found lurking in the most mundane of applications”. So embrace applied work but always look for the abstract deep problems – that summarizes my advice.
Ajay- What are the challenges of running data mining for a big big website.
Dr Fayyad- There are many challenges. Most algorithms will not work due to scale. Also, most of the problems have an unusually high dimensionality – so simple tricks like sampling won’t work. You need to be very clever on how to sample and how to reduce dimensionality by applying the right variable transformations.

The variety of problems is huge, and the fact that the Internet is evolving and growing rapidly, means that the problems are not fixed or stationary. A solution that works well today will likely fail in a few months – so you need to always innovate and always look at new approaches. Also, you need to build automated tools to help detect changes and address them as soon as they arise. 

Problems with 1000 10,000 or millions of variables are very common in web challenges. Finally, whatever you do needs to work fast or else you will not be able to keep up with the data flux. Imagine falling behind on processing 25 Terabytes of data per day. If you fall behind by two days, you will never be able to catch up again! Not within any reasonable budget constraint. So you try never to go down.
Ajay-      What are the 5 most important things that the data miner should avoid in doing analysis.

Dr Fayyad-I never thought about this in terms of top 5, but here are the big ones that come to mind, not necessarily in any order
a.       The algorithms knows nothing about the data, and the knowledge of the domain is in the head of the domain experts. As I always say, an ounce of knowledge is worth a ton of data – so seek and model what the experts know or your results will look silly
b.      Don’t let an algorithm fish blindly when you have lots of data. Use what you know to reduce the dimensionality quickly. The curse of dimensionality is never to be under-estimated
c.       Resist the temptation to cheat: selecting training and test sets can easily fool you into thinking you have something that works. Test it honestly against new data, never “peek” at the test data – what you see will force you to cheat without knowing it.
d.      Business rules typically dominate data mining accuracy, so be sure to incorporate the business and legal constraints into your mining.
e.       I have never seen a large database in my life that came from a static distribution that was sampled independently. Real databases grow to be big through lots of systematic changes and biases, and they are collected over years from changing underlying distribution: segmentation is a pre-requisite to any analysis. Most algorithms assume that data is IID (independent and identically distributed)

Ajay-   Do you think softwares like Hadoop and MapReduce will change the online database permanently. What further developments do you see in this area.

Dr Fayyad- I think they will (and have) changed the landscape dramatically, but they do not address everything. Many problems lend themselves naturally to Map-Reduce and many new approaches are enabled by Map-Reduce. However, there are many problems where M-R does not do much. I see a lot of problems being addressed by a large grid nowadays when they don’t need it. This is often a huge waste of computational resources. We need to learn how to deal with a mix of tools and platforms. I think M-R will be with us for a long time and will be a staple tool – but not a universal one.
Ajay-    I look forward to the day when I have just a low priced netbook and fast internet connection, and upload a Gigabyte of data and run advanced analytics on the browser. How far or soon do you think it is possible?
Dr Fayyad- Well, I thnk the day is already here. In fact, much of our web search today is conducted exactly in that model. A lot of web analysis, and much of scientific analysis is done like this today.
Ajay-    Describe some of the conferences you are currently involved with and the research areas that excites you the most.
Dr Fayyad- I am still very involved in knowledge discovery and data mining conferences (especially the KDD series), machine learning, some statistics, and some conferences on search and internet.  Most exciting conferences for me are ones that cover a mix of topics but that address real problems. Examples include understanding how social networks evolve and behave, understanding dimensionality reductions (like random projections in very high-D spaces) and generally any work that gives us insight into why a particular technique works better and where the open challenges are.
Ajay-  What are the next breakthrough areas in data mining. Can we have a  Google or Yahoo in fields of business intelligence as well given their huge market potential and uncertain ROI.
Dr Fayyad- We already have some large and healthy businesses in BI and quite a huge industry in consulting. If you are asking particularly about the tools market then I think that market is very limited. The users of analysis tools are always going to be small in number. However, once the BI and Data Mining tools are embedded in vertical applications, then the number of users will be tremendous. That’s where you will see success.
Consider the examples of Google or Yahoo! – and now Microsoft with BING search engine.  Search engines today would not be good without machine learning/data mining technology. In fact MLR (Machine Learned Ranking) is at the core of the ranking methodology that decides which search results bubble to the top of the list. The typical web query is 2.6 keywords long and has about a billion matches. What matters are the top 10. The function that determines these is a relevance ranking algrorithm that uses machine learning to tune a formula that considers hundreds or thousands of variables about each document. So in many ways, you have a great example of this technology being used by hundreds of millions of people every day – without knowing it!
Success will be in applications where the technology becomes invisible – much like the internal combustion engine in your car or the electric motor in your coffee grinder or power supply fan. I think once people start building verticalized solutions that embed data mining and BI, we will hit success. This already has happened in web search, in direct marketing, in advertising targeting, in credit scoring, in fraud detection, and so on…

Ajay-  What do you do to relax. What publications would you recommend for staying up to date for the data mining people especially the younger analysts.
Dr Fayyad- My favorite activity is sleep when I can get it J.  But more seriously, I enjoy reading books, playing chess, skiing (on water or snow – downhill or x-country), or any activities with my kids.  I swim a lot and that gives me much time to think and sort things out.
I think for keeping up with the technical advances in data mining: the KDD conferences, some of the applied analytics conferences, the WWW conferences, and the data mining journals. The ACM SIGKDD publishes a nice newsletter called SIGKDD explorations. It is free with a very low membership fee and it has a lot of announcements and survey papers on new topics and important areas (www.kdd.org).  Also, a good list to keep up with is an email list called KDNuggets edited by Gregory Piatetsky-Shapiro.
 

Biography (www.fayyad.com/usama )-

Usama Fayyad founded Open Insights (www.open-insights.com) to deliver on the vision of bridging the gap between data and insights and to help companies develop strategies and solutions not only to turn data into working business assets, but to turn the insights available from the growing amounts of data into critical components of an enterprise’s strategy for approaching markets, dealing with competitors, and acquire and retain customers.

In his prior role as Chief Data Officer of Yahoo! he built the data teams and infrastructure to manage the 25 terabytes of data per day that resulted from the company’s operations. He also built up the targeting systems and the data strategy for how to utilize data to enhance revenue and to create new revenue sources for the company.

In addition, he was the founding executive for Yahoo! Research, a scientific research organization that became the top research place in the world working on inventing the new sciences of the Internet.

Data professionals need to know about the data domain in addition to the data structure domain – Karen Lopez

Ajay- Describe your career in science. How would you persuade younger students to take more science courses.

Karen- I’ve always had an interest in science and I attribute that to the great science teachers I had. I studied information systems at Purdue University though a unique program that focuses on systems analysis and computer technologies. I’m one of the few who studied data and process modeling in an undergraduate program 25+ years ago.

I believe that it is very important that we find a way of attracting more scientists to teach. In both the natural and computer sciences, it’s difficult for institutions to tempt scientists away from professional positions that offer much greater compensation. So I support programs that find ways to make that happen.

Ajay- If you had to give advice to a young person starting their career in BI and had to give them advice in just three points – what would they be?

Karen- Wow. It’s tough to think of just three things, but these are recommendations that I make often:

- Remember that every design decision should be made based on cost, benefit, and risk. If you can’t clearly describe these for every side of a decision, then you aren’t doing design; you are guessing.

- No one beside you is responsible for advancing your skills and keeping an eye on emerging practices. Don’t expect your employer to lay out a career plan that is in your best interest. That’s not their job. Data professionals need to know about the data domain in addition to the data structure domain. The best database or data warehouse design in the world is worse than uses useless if the how the data is processed is wrong. Remember to expand your knowledge about data, not just the data structures and tools.

- All real-world work involves collaboration and negotiation. There is no one right answer that works for every situation. Building your skills in these areas will pay off significantly.

Ajay- What do you think is the best way for a technical consultant and client to be on the same page regarding requirements. Which methodology or template have you used, and which has given you the most success.

Karen- While I’m a huge fan of modeling (data modeling and other modeling), I still think that giving clients a prototype or mockup of something that looks real to them goes a long way. We need to build tools and competencies to develop these prototypes quickly. It’s a lost art in the data world.

Ajay- What are the special incentives that make Canada a great place for tech entrepreneurs rather than say go to the United States. ( Note- Disclaimer I have family in Canada and study in the US)

Karen- I prefer not to think of this as an either-or decision. I immigrated to Canada from the US about 15 years ago, but most of our business is outside of Canada. I have enjoyed special incentives here in Canada for small businesses as well as special programs that allowed me to work in Canada as a technical professional before I moved here permanently.

Overall, I have found Canadian employers more open to sponsoring foreign workers and it is easier for them to do so than what my US clients experience. Having said that, a significant portion of my work over the last few years has been on global projects where we leverage online collaboration tools to meet our goals. The advent of these tools has made it much easier to work from wherever I am and to work with others regardless of their visa statuses.

Where a company forms is less tied to where one lives or works these days.

Ajay- Could you tell us more about the Zachman framework (apart from the wikipedia reference)? A practical example on how you used it on an actual project would be great.

Karen- Of course the best resource for finding out about the Zachman framework is from John Zachman himself http://www.zachmaninternational.com/index.php/home-article/13 . He offers some excellent courses and does a great deal of public speaking at government and DAMA events. I highly recommend anyone interested in the Framework to hear about it directly from him.

There are many misunderstandings about John’s intent, such as the myth that he requires big upfront modeling (he doesn’t), that the Framework is a methodology (it isn’t), or that it can only be used to build computer systems (it can be used for more than that).

I have used the Zachman Framework to develop a joint Business-IT Strategic Information Systems Plan as well as to inventory and track progress of multi-project programs. One interesting use was a paper I authored for the Canadian Information Processing Society (CIPS) on how various educational programs, specializations, and certifications map to the Zachman Framework. I later developed a presentation about this mapping for a Zachman conference.

For a specific project, the Zachman Framework allows business to understand where their enterprise assets are being managed – and how well they are managed. It’s not an IT thing; it’s an enterprise architecture thing.

Ajay- What does Karen Lopez do for fun when not at work, traveling, speaking or blogging.

Karen- Sometimes it seems that’s all I do. I enjoy volunteering for IT-related organizations such as DAMA and CIPS. I participate in the accreditation of college and university educational programs in Canada and abroad. As a member of data-related standards bodies, namely the Association for Retail Technology Standards and the American Dental Association, I help develop industry standard data models. I’ve also been a spokesperson for a CIPS program to encourage girls to take more math and science courses throughout their student careers so that they may have access to great opportunities in the future.

I like to think of myself as a runner; last year I completed my first half marathon, which I’d never thought was possible. I am studying Hindi and Sanskrit. I’m also a addicted to reading and am thankful that some of it I actually get paid to do.

Biography

Karen López is a Senior Project Manager at InfoAdvisors, Inc. Karen is a frequent speaker at DAMA conferences and DAMA Chapters. She has 20+ years of experience in project and data management on large, multi-project programs. Karen specializes in the practical application of data management principles. Karen is also the ListMistress and moderator of the InfoAdvisors Discussion Groups at www.infoadvisors.com. You can reach her at www.twitter.com/datachick

Ajay- Describe your early science career. How would you encourage today’s generation to take up science and math careers?

John- I was a history major in college, but I graduated into a weak job market. So I went to graduate school and discovered statistics and computer science to be very captivating. Of course, I grew up in the moon-race science generation and was always a science enthusiast.

Ajay- Archimedes leapt out the bath shouting “Eureka” when he discovered his principle. Could you describe a “Eureka” moment while creating the SAS language when you and Jim Goodnight were working on it?

John- I think that the moments of discovery were more like “Oh, we were idiots” as we kept having to rewrite much of the product to handle emerging environments, like CMS, minicomputers, bitmap workstations, personal computers, Windows, client-server, and now the cloud. Several of the rewrites were even changing the language we implemented it in. But making the commitment to evolve led to an amazing sequence of growth that is still going on after 35 years.

Ajay- Describe the origins of JMP. What specific market segments does the latest release of JMP target?

John- JMP emerged from a recognition of two things: size and GUI. SAS’ enterprise footprint was too big a commitment for some potential users, and we needed a product to really take advantage of graphical interactivity. It was a little later that JMP started being dedicated more to the needs of engineering and science users, who are most of our current customers.

Ajay- What other non-SAS Institute software do you admire or have you worked with? Which areas is JMP best suited for? For which areas would you recommend software other than JMP to customers?

John- My favorite software was the Metrowerks CodeWarrior development environment. Sadly, it was abandoned among various Macintosh transitions, and now we are stuck with the open-source GCC and Xcode. It’s free, but it’s not as good.

JMP is perfect for anyone who wants to do exploratory data analysis and modeling in a visual and interactive way. This is something organizations of all kinds want to do. For analytics beyond what JMP can do, I recommend SAS, which has unparalleled breadth, depth and power in its analytic methods.

Ajay- I have yet to hear of a big academic push for JMP distribution in Asia. Are there any plans to distribute JMP for free or at very discounted prices in academic institutions in countries like India, China or even the rest of the USA?

John- We are increasing our investment in supporting academic institutions, but it has not been an area of strength for us. Professors seem to want the package they learned long ago, the language that is free or the spreadsheet program their business students already have. JMP’s customers do tell us that they wish the universities would train their prospective future employees in JMP, but the universities haven’t been hearing them. Fortunately, JMP is easy enough to pick up after you enter the work world. JMP does substantially discount prices for academic users.

Ajay- What are your views on tech offshoring, given the recession in the United States?

John- As you know, our products are mostly made in the USA, but we do have growing R&D operations in Pune and Beijing that have been performing very well. Even when the software is authored in the US, considerable work happens in each country to localize, customize and support our local users, and this will only increase as we become more service-oriented. In this recession, JMP has still been growing steadily.

Ajay-  What advice would you give to young graduates in this recession? How does learning JMP enhance their prospect of getting a job?

John- Quantitative fields have been fairly resistant to the recession. North Carolina State University, near the SAS campus, even has a Master of Science in Analytics < http://analytics.ncsu.edu/ > to get people job-ready. JMP experience certainly helps get jobs at our major customers.

Ajay- What does John Sall do in his free time, when not creating world-class companies or groovy statistical discovery software?

John- I lead the JMP division, which has been a fairly small part of a large software company (SAS), but JMP is becoming bigger than the whole company was when JMP was started. In my spare time, I go to meetings and travel with the Nature Conservancy <http://www.nature.org/ >, North Carolina State University <http:// http://ncsu.edu/ >, WWF <http://wwf.org/ >, CARE <http://www.care.org/ > and several other nonprofit organizations that my wife or I work with.

Official Biography

John Sall is a co-founder and Executive Vice President of SAS, the world’s largest privately held software company. He also leads the JMP business division, which creates interactive and highly visual data analysis software for the desktop.

Sall joined Jim Goodnight and two others in 1976 to establish SAS. He designed, developed and documented many of the earliest analytical procedures for Base SAS® software and was the initial author of SAS/ETS® software and SAS/IML®. He also led the R&D effort that produced SAS/OR®, SAS/QC® and Version 6 of Base SAS.

Sall was elected a Fellow of the American Statistical Association in 1998 and has held several positions in the association’s Statistical Computing section. He serves on the board of The Nature Conservancy, reflecting his strong interest in international conservation and environmental issues. He also is a member of the North Carolina State University (NCSU) Board of Trustees. In 1997, Sall and his wife, Ginger, contributed to the founding of Cary Academy, an independent college preparatory day school for students in grades 6 through 12.

Sall received a bachelor’s degree in history from Beloit College in Beloit, WI, and a master’s degree in economics from Northern Illinois University in DeKalb, IL. He studied graduate-level statistics at NCSU, which awarded him an honorary doctorate in 2003.

About JMP-

Originally nicknamed as John’s Macintosh Program, JMP is a leading software program in data visualization for statistical software. Researchers and engineers – whose jobs didn’t revolve solely around statistical analysis – needed an easy-to-use and affordable stats program. A new software product, today known as JMP®, was launched in 1989 to dynamically link statistical analysis with the graphical capabilities of Macintosh computers. Now running on all platforms, JMP continues to play an important role in modeling processes across industries as a desktop data visualization tool. It also provides a visual interface to SAS in an expanding line of solutions that includes SAS Visual BI and SAS Visual Data Discovery. Sall remains the lead architect for JMP.

Citation- http://www.sas.com/presscenter/bios/jsall.html

Ajay- I am thankful to John and his marketing communication specialist Arati for this interview.With an increasing focus on data to drive more rational decision making, SAS remains an interesting company to watch for in the era of mega- vendors and any SAS Institute deal and alliance will be  making potential investment bankers as well as newer customers drool. For previous interviews and coverage of SAS please use www.decisionstats.com/tag/sas

No one likes to feel blamed for causing or failing to fix the data quality problems- Jim Harris, Data Quality Expert.

Ajay- Why the name OCDQ? What drives your passion for data quality? Name any anecdotes where bad data quality really messed up a big BI project.

Jim Harris - Ever since I was a child, I have had an obsessive-compulsive personality. If you asked my professional colleagues to describe my work ethic, many would immediately respond: “Jim is obsessive-compulsive about data quality…but in a good way!” Therefore, when evaluating the short list of what to name my blog, it was not surprising to anyone that Obsessive-Compulsive Data Quality (OCDQ) was what I chose.

On a project for a financial services company, a critical data source was applications received by mail or phone for a variety of insurance products. These applications were manually entered by data entry clerks. Social security number was a required field and the data entry application had been designed to only allow valid values. Therefore, no one was concerned about the data quality of this field – it had to be populated and only valid values were accepted.

When a report was generated to estimate how many customers were interested in multiple insurance products by looking at the count of applications per social security number, it appeared as if a small number of customers were interested in not only every insurance product the company offered, but also thousands of policies within the same product type. More confusion was introduced when the report added the customer name field, which showed that this small number of highly interested customers had hundreds of different names. The problem was finally traced back to data entry.

Many insurance applications were received without a social security number. The data entry clerks were compensated, in part, based on the number of applications they entered per hour. In order to process the incomplete applications, the data entry clerks entered their own social security number.

On a project for a telecommunications company, multiple data sources were being consolidated into a new billing system. Concerns about postal address quality required the use of validation software to cleanse the billing address. No one was concerned about the telephone number field – after all, how could a telecommunications company have a data quality problem with telephone number?

However, when reports were run against the new billing system, a high percentage of records had a missing telephone number. The problem was that many of the data sources originated from legacy systems that only recently added a telephone number field. Previously, the telephone number was entered into the last line of the billing address.

New records entered into these legacy systems did start using the telephone number field, but the older records already in the system were not updated. During the consolidation process, the telephone number field was mapped directly from source to target and the postal validation software deleted the telephone number from the cleansed billing address.

Ajay- Data Quality – Garbage in, Garbage out for a project. What percentage of a BI project do you think gets allocated to input data quality? What percentage of final output is affected by the normalized errors?

Jim Harris- I know that Gartner has reported that 25% of critical data within large businesses is somehow inaccurate or incomplete and that 50% of implementations fail due to a lack of attention to data quality issues.

The most common reason is that people doubt that data quality problems could be prevalent in their systems. This “data denial” is not necessarily a matter of blissful ignorance, but is often a natural self-defense mechanism from the data owners on the business side and/or the application owners on the technical side.

No one likes to feel blamed for causing or failing to fix the data quality problems.

All projects should allocate time and resources for performing a data quality assessment, which provides a much needed reality check for the perceptions and assumptions about the quality of the data. A data quality assessment can help with many tasks including verifying metadata, preparing meaningful questions for subject matter experts, understanding how data is being used, and most importantly – evaluating the ROI of data quality improvements. Building data quality monitoring functionality into the applications that support business processes provides the ability to measure the effect that poor data quality can have on decision-critical information.

Ajay- Companies talk of paradigms like Kaizen, Six Sigma and LEAN for eliminating waste and defects. What technique would you recommend for a company just about to start a major BI project for a standard ETL and reporting project to keep data aligned and clean?

Jim Harris- I am a big advocate for methodology and best practices and the paradigms you mentioned do provide excellent frameworks that can be helpful. However, I freely admit that I have never been formally trained or certified in any of them. I have worked on projects where they have been attempted and have seen varying degrees of success in their implementation. Six Sigma is the one that I am most familiar with, especially the DMAIC framework.

However, a general problem that I have with most frameworks is their tendency to adopt a one-size-fits-all strategy, which I believe is an approach that is doomed to fail. Any implemented framework must be customized to adapt to an organization’s unique culture. In part, this is necessary because implementing changes of any kind will be met with initial resistance, but an attempt at forcing a one-size-fits-all approach almost sends a message to the organization that everything they are currently doing is wrong, which will of course only increase the resistance to change.

Starting with a framework as a reference provides best practices and recommended options of what has worked for other organizations. The framework should be reviewed to determine what can best be learned from it and to select what will work in the current environment and what simply won’t. This doesn’t mean that the selected components of the framework will be implemented simultaneously. All change comes gradually and the selected components will most likely be implemented in phases.

Fundamentally, all change starts with changing people’s minds. And to do that effectively, the starting point has to be improving communication and encouraging open dialogue. This means more of listening to what people throughout the organization have to say and less of just telling them what to do. Keeping data aligned and clean requires getting people aligned and communicating.

Ajay- What methods and habits would you recommend to young analysts starting in the BI field for a quality checklist?

Jim Harris- I always make two recommendations.

First, never make assumptions about the data. I don’t care how well the business requirements document is written or how pretty the data model looks or how narrowly your particular role on the project has been defined. There is simply no substitute for looking at the data.

Second, don’t be afraid to ask questions or admit when you don’t know the answers. The only difference between a young analyst just starting out and an expert is that the expert has already made and learned from all the mistakes caused by being afraid to ask questions or admitting when you don’t know the answers.

Ajay- What does Jim Harris do to have quality time when not at work?

Jim- Since I enjoy what I do for a living so much, it sometimes seems impossible to disengage from work and make quality time for myself. I have also become hopelessly addicted to social media and spend far too much time on Twitter and Facebook. I have also always spent too much of my free time watching television and movies. I do try to read as much as I can, but I have so many stacks of unread books in my house that I could probably open my own book store. True quality time typically requires the elimination of all technology by going walking, hiking or mountain biking. I do bring my mobile phone in case of emergencies, but I turn it off before I leave.

Biography-

Jim Harris is the Blogger-in-Chief here at Obsessive-Compulsive Data Quality (OCDQ), which is an independent blog offering a vendor-neutral perspective on data quality.

He is an independent consultant, speaker, writer and blogger with over 15 years of professional services and application development experience in data quality (DQ), and business intelligence (BI),

Jim has worked with Global 500 companies in finance, brokerage, banking, insurance, healthcare, pharmaceuticals, manufacturing, retail, telecommunications, and utilities. Jim also has a long history with the product that is now known as IBM InfoSphere QualityStage. Additionally, he has some experience with Informatica Data Quality and DataFlux dfPower Studio.

Jim can be followed at twitter.com/ocdqblog and contacted at http://www.ocdqblog.com/contact/

Ajay- What does this round of Predictive Analytics World have —–which was not there in the edition earlier in the year.

Eric- Predictive Analytics World (pawcon.com) – Oct 20-21 in DC delivers a fresh set of 25 vendor-neutral presentations across verticals employing predictive analytics, such as banking, financial services, e-commerce, education, healthcare, high technology, insurance, non-profits, publishing, retail and telecommunications.

PAW features keynote speaker, Stephen Baker, author of The Numerati and Senior writer at BusinessWeek.  His keynote is described at www.predictiveanalyticsworld.com/dc/2009/agenda.php#day2-2

A strong representation of leading enterprises have signed up to tell their stories — speakers will present how predictive analytics is applied at Aflac, AT&T Bell South, Amway, The Coca-Cola Company, Financial Times, Hewlett-Packard, IRS, National Center for Dropout Prevention, The National Rifle Association, The New York Times, Optus (Australian telecom), PREMIER Bankcard, Reed Elsevier, Sprint-Nextel, Sunrise Communications (Switzerland), Target, US Bank, U.S. Department of Defense, Zurich — plus special examples from Anheuser-Busch, Disney, HSBC, Pfizer, Social Security Administration, WestWind Foundation and others.

To see the entire agenda at a glance: www.predictiveanalyticsworld.com/dc/2009/agenda_overview.php

We’ve added a third workshop, offered the day before (Oct 19), “Hands-On Predictive Analytics.  There’s no better way to dive in than operating real predictive modeling software yourself – hands-on.”  For more info: www.predictiveanalyticsworld.com/dc/2009/handson_predictive_analytics.php

Ajay- What do academics, corporations and data miners gain in this conference? list 4 bullet points for the specific gains.

Eric- A. First, PAW’s experienced speakers provide the “how to” of predictive analytics. PAW is a unique conference in its focus on the commercial deployment of predictive analytics, rather than research and development. The core analytical technology is established and proven, valuable as-is without additional R&D — but that doesn’t mean it’s a “cakewalk” to employ it successfully to ensure value is attained.  Challenges include data requirements and preparation, integration of predictive models and their scores into existing organizational systems and processes, tracking and evaluating performance, etc. There’s no better way to hone your skills and cultivate an informed plan for your organization’s efforts than hearing how other organizations did it.

B. Second, PAW covers the latest state-of-the-art methods produced by research labs, and how they provide value in commercial deployment. This October, almost all sessions in Track 2 are at the Expert/Practitioner-level.  Advanced topics include ensemble models, uplift modeling (incremental modeling), model scoring with cloud computing, predictive text analytics, social network analysis, and more.

PAW’s pre- and post-conference workshops round out the learning opportunities. In addition to the hands-on workshop mentioned above, there is a course covering core methods, “The Best and the Worst of Predictive Analytics: Predictive Modeling Methods and Common Data Mining Mistakes” (www.predictiveanalyticsworld.com/dc/2009/predictive_modeling_methods.php) and a business-level seminar on decision automation and support, “Putting Predictive Analytics to Work” (www.predictiveanalyticsworld.com/dc/2009/predictive_analytics_work.php).

C. Third, the leading predictive analytics software vendors and consulting firms are present at PAW as sponsors and exhibitors, available to provide demos and answer your questions.  What do the predictive analytics solutions do, how do they compare, and which is best for your? PAW is the one-stop-shop for selecting the tool or solution most suited to address your needs.

D. Fourth, PAW provides a unique, focused opportunity to network with colleagues and establish valuable contacts in the predictive analytics industry.  Mingle, connect and hang out with professionals facing similar challenges (coffee breaks, meals, and at the reception).

Ajay- How do you balance the interests of various competing softwares and companies who sponsor such event?

Eric- As a vendor-neutral event, PAW’s core program of 25 sessions is booked exclusively with enterprise practitioners, thought leaders and adopters, with no predictive analytics software vendors speaking or co-presenting. These sessions provide substantive content with take-aways which provide value that’s independent of any particular software solution — no product pitches!  Beyond these 25 sessions are three short sponsor sessions that are demarcated as such, and, despite being branded, generally prove to be quite substantive as well.  In this way, PAW delivers a high quality, unbiased program.

Supplementing this vendor-neutral program, the room right next door has an expo where attendees have all the access to software and solution vendors they could want (cf. in my answer to the prior question regarding software vendors, above).

Here are a couple more PAW links:

For informative PAW event updates:
www.predictiveanalyticsworld.com/notifications.php

To sign up for the PAW group on LinkedIn, see:
www.linkedin.com/e/gis/1005097

Ajay- Describe your career in science including research that you specialize in. How would you motivate students today to go for science careers

Eric- Well, first off, my work as a predictive analytics consultant, instructor and conference chair is in the application of established technology, rather than the research and development of new or improved methods.

But the Ph.D. next to my name reveals my secret past as an “academic”. Pure research is something I really enjoyed and I kind of feel like I had a brain transplant in order to change to “real world work”. I’m glad I made the change, although I see good sides to both types of work (really, they’re like two entirely different lifestyles).

In my research I focused on core predictive modeling methods. The ability for a computer to automatically learn from experience (data really is recorded experience, after all), is the best thing since sliced bread. Ever since I realized, as a kid, that space travel would in fact be a huge pain in the neck, nothing in science has ever seemed nearly as exciting.

Predictive analytics is an endeavor in machine learning. A predictive model is the encoding of a set of rules or patterns or regularities at some level. The model is the thing output by automated, number-crunchin’ analysis and, therefore, is the thing “learned” from the “experience” (data).  The “magic” here is the ability of these methods to find a model that performs not only over the historical data on your disk drive, but that will perform equally well for tomorrow’s new situations. That ability to generalize from the data at hand means the system has actually learned something.

And indeed the ability to learn and apply what’s been learned turns out to provide plenty of business value, as I imagined back in the lab.  The output of a predictive model is a predictive score for each individual customer or prospect.  The score in turn directly informs the business decision to be taken with that individual customer (to contact or not to contact; with which offer to contact, etc.) – business intelligence just doesn’t get more actionable than that.

For the impending student, I’d first point out the difference between applied science and research science. Research science is fun in that you have the luxury of abstraction and are usually fairly removed from the need to prove near-term industrial applicability. Applied science is fun for the opposite reason: The tangle of challenges, although less abstract and in that sense more mundane, are the only thing between you and getting the great ideas of the world to actually work, come to fruition, and have an irrefutable impact.

Ajay- What are the top five conferences in analytics and data mining in your opinion in the world including PAW.

Eric- KDD – The leading event for research and development of the core methods behind the commercial deployments covered at PAW (“Knowledge Discovery and Data Mining”).

ICML – Another long-standing research conference on machine learning (core data mining).

eMetrics.org – For online marketing optimization and web analytics

Text Analytics Summit – Text mining can leverage “unstructured data” (text) to augment predictive analytics; the chair of this conference is speaking at PAW on just that topic: www.predictiveanalyticsworld.com/dc/2009/agenda.php#day2-15

Predictive Analytics World, the business-focused event for predictive analytics professionals, managers and commercial practitioners – focused on the commercial deployment of predictive analytics: pawcon.com

Ajay- Would PAW 2009 have video archives, videos as well or podcasts for people not able to attend on site.

Eric- While the PAW conferences emphasize in-person participation, we are in the planning stages for future webcasts and other online content. PAW’s “Predictive Analytics Guide” has a growing list of online resources: www.predictiveanalyticsworld.com/predictive_analytics.php

Ajay- How do you think social media marketing can help in these conferences.

Eric- Like most events, PAW leverages social media to spread the word.

But perhaps most pertinent is the other way around: predictive analytics can help social media by increasing relevancy, dynamically selecting the content to which each reader or viewer is most likely to respond.

Ajay- Do you have any plans to take PAW more international? Any plans for a PAW journal for trainings and papers.

Eric- We’re in discussions on these topics, but for now I can only say, stay tuned!

Biographyy

The president of Prediction Impact, Inc., Eric Siegel is an expert in predictive analytics and data mining and a former computer science professor at Columbia University, where he won awards for teaching, including graduate-level courses in machine learning and intelligent systems – the academic terms for predictive analytics.He has published 13 papers in data mining research and computer science education, has served on 10 conference program committees, has chaired a AAAI Symposium held at MIT, and is the founding chair of Predictive Analytics World.

For more on Predictive Analytic World-

Predictive Analytics World Conference
October 20-21, 2009, Washington, DC
www.predictiveanalyticsworld.com
LinkedIn Group: www.linkedin.com/e/gis/1005097

Ajay- Describe your career in science starting from college till today. How would you interest young students in science careers today in the mid of the recession

Gary - I knew that I wanted to be in “Science” even before college days, taking all the science and math courses I could in high school. This continued in undergraduate college years at a private college [Hamline University, St. Paul, Minnesota……..older than the State of Minnesota, founded in 1854, and had the first Medical School, later “sold” to the University of Minnesota] as a Biology and Chemistry major, with a minor in education. From there is did a M.S. conducting a “Physiological genetics research project”, and then a Ph.D. at another institution where I worked on Genetic Polymorphisms of Mouse blood enzymes. So through all of this, I had to use statistics to analyze the data. My M.S. was analyzed before the time of even “electronic calculators”, so I used, if you can believe this, a “hand cranked calculator”, rented, one summer to analyze my M.S. dataset. By the time my Ph.D. thesis data was being analyzed, electronic calculators were available, but the big main-frame computers were on college campuses, so I punched the data into CARDS, walked down the hill to the computing center, dropped off the stack of cards, to come back the next day to get “reams of output” on large paper [about 15” by 18”, folded in a stack, if anyone remembers those days …]. I then spent about 30 years doing medical research in academic environments with the emphasis on genetics, biochemistry, and proteonomics in the areas of mental illness and Alzheimer’s Disease, which became my main area of study, publishing the first book in 1989 on the GENETICS OF ALZHEIMER’S DISEASE.

Today, in my “semi-retirement careers”, one side-line outreach is working with medical residents on their research projects, which I’ve been doing for about 7 or 8 years now. This involves design of the research project, data collection, and most importantly “effective and accurate” analysis of the datasets. I find this a way I can reach out to the younger generation to interest them not only in “science”, but in doing “science correctly”. As you probably know, we are in the arena of the “Duming of America”; anti-science, if you wish. I’ve seen this happening for at least 30 years, during the 1980’s, 1990’s, and continuing into this Century. Even the medical residents I get to work with each year have been going “downhill” yearly in their ability to “problem solve”. I believe this is an effect of this “dumning of America”.

There are several books coming out on this Dumning of America this summer; one the first week of June, another on July 12, and another in September [see the attached PPT for slides with the covers of these 3 books}. It is a real problem, as Americans over the past few decades have moved towards “wanting simple answers”, and most things in the “real world”, e.g. reality are not simple………..that’s where Science comes in.

A recent 2008 study done by the School of Public Health at Ohio University showed that up to 88% of the published scientific papers in a top respected cancer journal either used statistics INCORRECTLY, and/or the CONCLUSION was INCORRECT. When I and my wife both did Post-Docs in Psychiatric Epidemiology in 1980-82, basically doing an MPH, the first words out of the mouth of the “Biostats – Epidemiology” professor in the first lecture to the incoming MPH students was “We might as well through out most of the medical research literature of the past 25 years, as it has either not been designed correctly or statistics have been used incorrectly”!!! ……That caught my attention. And following medical research [and medicine in general] I can tell you that “not much has changed in the past 25 years since then”, and thus that puts us “50 years behind in medical research” and medicine. ANALOGY: If some of our major companies, that are successfully using predictive analytics to organize and efficiently run their organizations, took on the “mode of operation” of medicine and medical research, they’d be “bankrupt” in 6 months” …. That’s what I tell my students.

Ajay- Describe some of the exciting things data mining can do to lower health care costs and provide more people with coverage.

Gary- As mentioned above, my personal feeling is that “medicine / health care” is 50 years “behind the times”, compared to the efficiency needed to successfully survive in this Global Economy; corporations and organizations like Wal-Mart, INTEL, many of our Pharmaceutical Companies, have used data mining / predictive analytics to survive successfully. Wal-Mart especially: Wal-Mart has it’s own set of data miners, and were writing their own procedures in the early 1990’s ………..before most of us ever heard of data mining; that is why Wal-Mart can go into China today, and open a store in any location, and know almost to 99% accuracy 1) how many check out stand needed, 2) what products to stock, 3) where in the store to stock them, and 4) what their profit margin will be. They have done this through very accurate “Predictive Analytics” modeling.

Other “ingrained” USA corporations have NOT grabbed onto this “most accurate” technology [e.g. predictive analytics modeling], and reaping the “rewards” of impending bankruptcy and disappearance today. Examples in the news, of course, our our 3 – big automakers in Detroit. If they had engaged effective data mining / modeling in the late 1990’s they could have avoided their current problems. I see the same for many of our oldest and larges USA Insurance Companies………..they are “middle management fat”, and I’ve seen their ratings go down over the past 10 years from an A rating to even a C rating [for the company in which I have my auto insurance ? you might ask me why I stay? …. An agent who is a friend, BUT it is frustrating, and this companies “mode of operation” is completely “customer un-friendly”.], while new insurance companies have “grabbed” onto modern technology, and are rising stars.

So my influence on the younger generation is to have my students do research and DATA ANALYSIS correctly.

Ajay- Describe your book ” HANDBOOK OF STATISTICAL ANALYSIS & DATA MINING APPLICATIONS”. Who would be the target audience of this and can corporate data miners gain from it as well.

Gary- There are several target audiences: The main audience we were writing for, after our Publisher looked at what “niches” had been un-met in data mining literature, was for the professional in smaller and middle sized businesses and organizations that needed to learn about “data mining / predictive analytics” “fast”…..e.g. maybe situations where the company did not have a data anlaysis group using predictive analytics, but the CEO’s and Professionals in the company knew they needed to learn and start using predictive analytics to “stay alive”. This seemed like potentially a very large audience. The book is oriented so that one does NOT have to start at chapter 1, and read sequentially, but instead can START WITH A TUTORIAL. Working through a tutorial, I’ve found in my 40 years of being in education, is the fastest way for a person to learn something new. And this has been confirmed………..I;ve had newcomers to data mining, who have already gotten the HANDBOOK, write me and say: “I’ve gone through a bunch of tutorials, and finding that I am really learning ‘how to do this’……..I’ve ready other books on ‘theory’, but just didn’t get the ‘hang of it’ from those”. My data mining consultants at StatSoft, who travel and work in “real world” situations every day, and who wrote maybe 1/3 of the tutorials in the HANDBOOK, tell me: “A person can go through the TUTORIALS in the HANDBOOK, and know 70% of what we who are doing predictive analytics consulting every day know !!!”

But there are other audiences: Corporate data miners can find it very useful also, as a “way of thinking as a data miner” can be gained from reading the book, as was expressed by one of the Amazon.com 5-STAR reviews: “What I like about this book is that it embeds those methods in a broader context, that of the philosophy and structure of data mining, especially as the methods are used in the corporate world. To me, it was really helpful in thinking like a data miner, especially as it involves the mix of science and art.”

But we’ve had others who have told us they will use is as an extra textbook in their Business Intelligence and Data Minng courses, because of the “richness” of the tutorials. Here’s a comment on the Amazon reviews from a Head of Business School who has maybe over 100 graduate students doing data mining:

“5.0 out of 5 stars. At last, a useable data mining book”

This is one of the few, of many, data mining books that delivers what it promises. It promises many detailed examples and cases. The companion DVD has detailed cases and also has a real 90 day trial copy of Statistica. I have taught data mining for over 10 years and I know it is very difficult to find comprehensive cases that can be used for classroom examples and for students to actually mine data. The price of the book is also very reasonable expecially when you compare the quantity and quality of the material to the typical intro stat book that usually costs twice as much as this data mining book.

The book also addresses new areas of data mining that are under development. Anyone that really wants to understand what data mining is about will find this book infinitively useful.”

So, I think the HANDBOOK will see use in many college classrooms.

Ajay- A question I never get the answer to is which data mining tool is good for what and not so good for what. Could you help me out with this one? What in your opinion, among the data mining and statistical tools used by you in your 40 years in this profession would you recommend for some uses, and what would you not recommend for other uses ( eg SAS,SPSS,KXEN,Statsoft,R etc etc)

Gary- This is a question I can’t answer well; but my book co-author, Robert Nisbet, Ph.D. can. He has used most of these softwares, and in fact has written 2 reviews over the past 6 years in which most of these have been discussed. I like “cutting edge endeavors”, that has been the modus operandi of my ‘career’, so when I took this “semi-retirement postion” as a data mining consultant at StatSoft, I was introduced to DATA MINING, as we started developing STATISTICA Data Miner shortly after I arrived. So most of my experience is with STATISTICA Data Miner, which of course has always been rated NO 1 in all the reviews on data miner software done by Dr. Nisbet – I believe this is primarily due to the fact that STATISTICA was written for the PC from the beginning, thus dos not have any legacy “main frame computer” coding in its history, and secondly StatSoft has been able to move rapidly to make changes as business and government data analysis needs change, and thirdly and most importantly, STATISTICA products have very “open architecture”, “flexibility”, and “customization” with every “built together / workable together” as one package. And of course the graphical output is second to none – that is how STATISTICA originally got its reputation. So I find no need of any other software, as if I need a new algorithm, I can program it to work with the “off the shelf” STATISTICA Data Miner algorithms, and thus get anything I need with the full graphical and other outputs seamlessly available.

Ask Bob Nisbet to answer this question, as he has the background to do so.

Ajay- What are the most interesting trends to watch out for in 2009-2010 in data mining in your opinion.

Gary- Things move so rapidly in this 21st century world, that this is difficult to say. Let me answer this with “hindsight”:

In late October, 2008 I wrote the first draft of Chapter 21 for the HANDBOOK. This was the “future directions of data mining”. You can look in that chapter yourself to find the 4 main areas I decided to focus on. One was on “social networking”, and one of the new examples used was TWITTER. At that time, less than one year ago, no one knew if TWITTER was going to amount to much or not ??? big question? Well, on Jan 14 when the US-AIRWAYS A320 Airbus made an emergency landing in the Hudson River, I got an EMAIL automatic message from CNN [that I subscribe to] telling me that a “plane was down in the Hudson, watch it live” …………I click on the live video: The voice form the Helicopter overhead was saying: “We see a plane, half sunk into the water, but no people? What has happened to the people? Are they all dead?………” Well, as it turned out, the CNN Helicopters had spend nearly one hour searching the river for the plane, as had other news agencies. BUT THE “ENTIRE” WORLD ALREADY KNEW !!! … Why? A person on a ferry that was crossing the river close to the crash landing used his I-Phone, snaped a photo, uploaded it to TWIT-PIX and sent a TWITTER message, and this was re-tweeted around the world. The world knew in “seconds to minutes” to which the traditional NEWS MEDIA was 1 hour late on the scene, when ALL the PEOPLE had been rescued and were on-shore in a warm building within 45 minutes of the landing. THE TRADITIONAL NEWS MEDIA ARRIVED 15 MINUTES AFTER EVERYTHING HAD HAPPENED !!!! ………AT THIS POINT we ALL KNEW that TWITTER was a new phenomenon ……….and it started growing, with 10,00 people an hour joining at one point in last spring of this year, and who knows what the rate is today. TWITTER has become a most important part not only of “social networking” among friends, but for BUSINESS —- companies even sending out ‘Parts Availability” lists to their dealers, etc.

TWITTER affected Chapter 21…………..I immediately re-wrote Chapter 21, including this first photo of the Hudson Plane crash-landing with all the people standing on the wings. BUT, not the end of this story: By the time the book was about to go to press, TWITTER had decided that “ownership” of uploaded photos resided with the photographer, and the person who took this original US-AIRBUS – PEOPLE ON THE WINGS photo wanted $600 for us to publish it in the HANDBOOK. So, I re-wrote again [the chapter was already “set” in page proofs……….so we had to make the changes directly at the printer]………this time finding another photo uploaded to social media, but in this case the person had “checked” the box to put the photo in public domain.

So TWITTER is one that I predicted would become important, but I’d thought it would be months AFTER the HANDBOOK was released in May, not last January!!!

Other things we presented in Chapter 21 about the “future of data mining” involved “photo / image recognition”, among others. The “Image Recognition”, and more importantly “movement recognition / analysis” for things like Physical Therapy and other medical areas may be more slow to evolve and fully develop, but are immensely important. The ability to analyze such “Three-dimensional movement data” is already available in rudimentary form in our version 9 of STATISTICA [just released in June], and anyone could implement it fully with MACROS, but it probably will be some time before it is fully feasible from a business standpoint to develop it with fully automatic “point and click” functionality to make it readily accessible for anyone’s use.

Ajay What would your advice be to a young statistician just starting his research career.

Gary- Make sure you delve in / grab in FULLY to the subject areas……….you need to know BOTH the “domain” of the data you are working with, and “correct” methods of data analysis, especially when using the traditional p-value statistics. Today’s culture is too much on “superficiality”………..good data analysis requires “depth” of understanding. One needs to FOCUS ………good FOCUS can’t be done with elaborate “multi-tasking”. Granted, today’s youth [the “Technology-Inherited”] probably have their brains “wired differently” than the “Technology-Immigrants” like myself [e.g. the older generations], but never-the-less, I see ERRORS all over the place in today’s world, from “typos” in magazine and newspaper, to web page paragraphs, links that don’t work, etc etc ……….and I conclude that this is all do to NON-FOCUSED / MULTI-TASKING people. You can’t drive a car / bus / train and TEXT MESSAGE at the same time ……….the scientific tests that have been conducted show that it takes 20-times as long for a TEXT MESSAGING driver to stop, than a driver fully focused on the road, when given a “danger” warning. [Now, maybe this scientific experiment used ALL TECHNOLOGY-IMMIGRANTS as drivers?? If so, the scientific design was “flawed” ……..they should have used BOTH Technology-Immigrants and Technology-Inheritants as participants in the study. Then we’d have 2 dependent, or target variables: Age and TEXT MESSAGING…..]

Short Bio-

Professor, 30 years medical research in genetics, DNA, Proteins, Neuropsychology of Schizophrenia and Alzheimer’s Disease……….now semi-retired position as DATA MINING CONSULTANT – SENIOR STATISTICIAN

Too many CRM systems are not usable. They are built by engineers that think of the system as a large database and the systems often look like a database making it difficult to use by the sales, support, and marketing people.

-John F Moore

Ajay – Describe your career journey from college to CTO. What changes in mindset did you undergo along the journey? What advice would you give to young students to take up science careers ?

John- First, I wanted to take time to thank you for the interview offer. I graduated from Boston University in 1988 with a degree in Electrical Engineering. At the time of my graduation I found myself to be very interested in the advanced taking place on the personal computing front by companies like Lotus with their 1-2-3 product. I knew that I wanted to be involved with these efforts and landed my first job in the software space as a Software Quality Engineer working on 1-2-3 for DOS.

I spent the first few years of my career working at Lotus as a developer, a quality engineer, and manager, on products such as Lotus 1-2-3 and Lotus Notes. Throughout those early career years I learned a lot and focused on taking as many classes as possible.

From Lotus I sought out the start-up environment and by early 2000 and joined a startup named Brainshark (http://www.brainshark.com). Brainshark was, and is, focused on delivering an asynchronous communication platform on the web and was one of the early providers of SAAS. In my seven years at Brainshark I learned a lot about delivering an Enterprise class SAAS solution on top of the Microsoft technology stack. The requirements to pass security audits for Fortune 500 companies, the need to match the performance of in-house solutions, resulted in all of us learning a great deal. These were very fun times.

I now work as the VP of Engineering and CTO at Swimfish, a services and software provider of business solutions. We focus on the financial marketplace where we have the founder has a very deep background, but also work within other verticals as well. Our products are focused on the CRM, document management, and mobile product space and are built on the Microsoft technology stack. Our customers leverage both our SAAS and on-premise solutions which require us to build our products to be more flexible than is generally required for a SAAS-only solution.

The exciting thing for me is the sheer amount of opportunities I see available for science/engineering students graduating in the near future. To be prepared for these opportunities, however, it will be important to not just be technically savvy.

Engineering students should also be looking at:

* Business classes. If you want to build cool products they must deliver business value.

* Writing and speaking classes. You must be able to articulate your ideas or no one will be willing to invest in them.

I would also encourage people to take chances, get in over your head as often as possible.You may fail, you may succeed. Either way you will gain experiences that make it all worthwhile.

Ajay- How do you think social media can help with CRM. What are the basic do’s and don’ts for social media CRM in your opinion?

John- You touch upon a subject that I am very passionate about. When I think of Social CRM I think about a system of processes and products that enable businesses to actively engage with customers in a manner that delivers maximum value to all. Customers should be able to find answers to their questions with minimal friction or effort; companies should find the right customers for their products.

Social CRM should deliver on some of these fronts:

* Analyze the web of relationships that exists to define optimal pathways. These pathways will define relationships that businesses can leverage for finding their customers. These pathways will enable customers to quickly find answers to their questions. For example, I needed an answer to a question about SharePoint and project management. I asked the question on Twitter and within 3 minutes had answers from two different people. Not only did I get the answer I needed but I made two new friends who I still talk to today.

* Monitor conversations to gauge brand awareness, identify customers having problems or asking questions. This monitoring should not be stalking; however, it should be used to provide quick responses to customers to benefit the greater community.

* Usability. Too many CRM systems are not usable. They are built by engineers that think of the system as a large database and the systems often look like a database making it difficult to use by the sales, support, and marketing people.

Finally, when I think of social media I think of these properties:

* Social is about relationship building.

* You should always add more value to the community than you take in return.

* Be transparent and honest. People can tell when you’re not.

Ajay-  You are involved in some noble causes – like using blog space for out of work techies and separately for Alzheimer’s disease. How important do you think is for people especially younger people to be dedicated to community causes?

John- My mother-in-law was diagnosed with Alzheimer’s disease at the age 57. My wife and I moved into their two-family house to help her through the final years of her life. It is a horrible disease and one that it is easy to be passionate about if you have seen it in action.

My motivation on the job front is very similar. I have seen too many people suffer through these poor economic times and I simply want to do what I can to help people get back to work.

It probably sounds corny, but I firmly believe that we must all do what we can for each other. Business is competitive, but it does not mean that we cannot, or should not, help each other out. I think it’s important for everyone to have causes they believe in. You have to find your passions in life and follow them. Be a whole person and help change the world for the better.

Ajay- Describe your daily challenges as head of Engineering of Swimfish, Inc How important is it for the tech team to be integrated with the business and understand it as well.

John- The engineering team at Swimfish works very closely with the business teams. It is important for the team to understand the challenges our customers are encountering and to build products that help the customer succeed. I am not satisfied with the lack of success that many companies encounter when deploying a CRM solution.

We go as deep as possible to understand the business, the processes currently in use, the disparate systems being utilized, and then the underlying technologies currently in use. Only then do we focus on the solutions and deliver the right solution for that company.

On the product front it is the same. We work closely with customers on the features we are planning to add, trying to ensure that the solutions meet their needs as well as the needs of the other customers in the market that we are hoping to serve.

I do expect my engineers to be great at their core job, that goes without question. However, if they cannot understand the business needs they will not work for me very long.My weeks at Swimfish always provide me with interesting challenges and opportunities.

My typical day involves:

* Checking in with our support team to understand if there are any major issues being encountered by any of our customers.

* Challenging the support team to hit their targets. I love sales as without them I cannot deliver products.

* Checking in with my developers and test teams to determine how each of our projects is doing. We have a daily standup as well, but I try and personally check-in with as many people as possible.

* Most days I spend some time developing, mostly in C#. My current focus area is on our next release of our Milestone Tracking Matrix where I have made major revisions to our user interface.

I also spend time interacting on various social platforms, such as Twitter, as it is critical for me to understand the challenges that people are encountering in their businesses, to keep up with the rapid pace of technology, and just to check-in with friends. Keep it real.

Ajay-  What are your views on off shoring work especially science jobs which ultimately made science careers less attractive in the US- at the same time outsourcing companies ( in India) generally pay only 1/3 rd of billing fees to salaries. Do you think concepts like ODesk can help change the paradigm of tech out-sourcing.

John- I have mixed opinions on off-shoring. You should not offshore because of perceived cost savings only. On net you will generally break even, you will not save as much as you might originally think.

I am, however, close to starting a relationship with a good development provider in Costa Rica. The reason for this relationship is not cost based, it is knowledge based. This company has a lot of experience with the primary CRM system that we sell to customers and I have not been successful in finding this experience locally. I will save a lot of money in upfront training on this skill-set; they have done a lot of work in this area already (and have great references). There is real value to our business, and theirs.

Note that Swimfish is already working with a geographically dispersed team as part of the engineering team is in California and part is in Massachusetts. This arrangement has already helped us to better prepare for an offshore relationship and I know we will be successful when we begin.

Ajay- What does John Moore do to have fun when he is not in front of his computer or with a cause.

John- As the father of two teenage daughters I spend a lot of time going to soccer, basketball, and softball games. I also enjoy spending time running, having completed a couple of marathons, and relaxing with a good book. My next challenge will be skydiving as my 17 year old daughter and I are going skydiving when she turns 18.

Brief Bio:

For the last decade I have worked as a senior engineering manager for SAAS applications built upon the Microsoft technology stack. I have established the processes, and hired the teams that delivered hundreds of updates ranging from weekly patches to longer running full feature releases. My background as a hands-on developer combined with my strong QA background has enabled me to deliver high quality software on-time.

You can learn more about me, and my opinions, by reading my blog at http://johnfmoore.wordpress.com/ or joining me on Twitter at http://twitter.com/JohnFMoore

I am a firm believer that the true benefits of BI are only realised when it leads to cultural transformation. -Peter James Thomas

Ajay- Describe about your early career including college to the present.

Peter -I was an all-rounder academically, but at the time that I was taking public exams in the 1980s, if you wanted to pursue a certain subject at University, you had to do related courses between the ages of 16 and 18. Because of this, I dropped things that I enjoyed such as English and ended up studying Mathematics, Further Mathematics, Chemistry and Physics. This was not because I disliked non-scientific subjects, but because I was marginally fonder of the scientific ones. In a way it is nice that my current blogging allows me to use language more.

The culmination of these studies was attending Imperial College in London to study for a BSc in Mathematics. Within the curriculum, I was more drawn to Pure Mathematics and Group Theory in particular, and so went on to take an MSc in these areas. This was an intercollegiate course and I took a unit at each of King’s College and Queen Mary College, but everything else was still based at Imperial. I was invited to stay on to do a PhD. It was even suggested that I might be able to do this in two years, given my MSc work, but I decided that a career in academia was not for me and so started looking at other options.

As sometimes happens a series of coincidences and a slice of luck meant that I joined a technology start-up, then called Cedardata, late in 1988; my first role was as a Trainee Analyst / Programmer. Cedardata was one of the first organisations to offer an Accounting system based on a relational database platform; something that was then rather novel, at least in the commercial arena. The RDBMS in question was Oracle version 5, running on VAX VMS – later DEC Ultrix and a wide variety of other UNIX flavours. Our input screens were written in SQL*Forms 2 – later Oracle Forms – and more complex processing logic and reports were in Pro*C; this was before PL/SQL. Obviously this environment meant that I had to become very conversant with SQL*Plus and C itself.

When I joined Cedardata, they had 10 employees, 3 customers and annual revenue of just £50,000 ($80,000). By the time I left the company eight years later, it had grown dramatically to having a staff of 250, over 300 clients in a wide range of industries and sales in excess of £12 million ($20 million). It had also successfully floatated on the main London Stock Exchange. When a company grows that quickly the same thing tends to happen to its employees.

Cedardata was probably the ideal environment for me at the time; an organisation that grew rapidly, offering new opportunities and challenges to its employees; that was fiercely meritocratic; and where narrow, but deep, technical expertise was encouraged to be rounded out by developing more general business acumen, a customer-focused attitude and people-management skills. I don’t think that I would have learnt as much, or progressed anything like as quickly in any other type of organisation.

It was also at Cedardata that I had my first experience of the class of applications that later became known as Business Intelligence tools. This was using BusinessObjects 3.0 to write reports, cross-tabs and graphs for a prospective client, the UK Foreign and Commonwealth Office (State Department). The approach must have worked as we beat Oracle Financials in a play-off to secure the multi-million pound account.

During my time at Cedardata, I rose to become an executive and filled a number of roles including Head of Development and also Assistant to the MD / Head of Product Strategy. Spending my formative years in an organisation where IT was the business and where the customer was King had a profound impact on me and has influenced my subsequent approach to IT / Business alignment.

Ajay- How would you convince young people to take maths and science more? What advice would you give to policy makers to promote more maths and science students?

Peter- While I have used little of my Mathematics directly in my commercial career, the approach to problem-solving that it inculcated in me has been invaluable. On arriving at University, it was something of a shock to be presented with Mathematical problems where you couldn’t simply look up the method of solution in a textbook and apply it to guarantee success. Even in my first year I had to grapple with challenges where you had no real clue where to start. Instead what worked, at least most of the time, was immersing yourself in the general literature, breaking down the problem into more manageable chunks, trying different techniques – sometimes quite recherché ones – to make progress, occasionally having an insight that provides a short-cut, but more often succeeding through dogged determination. All of that sounds awfully like the approach that has worked for me in a business context.

Having said that, I was not terribly business savvy as a student. I didn’t take Mathematics because I thought that it would lead to a career, I took it because I was fascinated by the subject. As I mentioned earlier, I enjoyed learning about a wide range of things, but Science seemed to relate to the most fundamental issues. Mathematics was both the framework that underpinned all of the Sciences and also offered its own world where astonishing an beautiful results could be found, independent of any applicability; although it has to be said that there are few braches of Mathematics that have not be applied somewhere or other.

I think you either have this appreciation of Science and Mathematics or you don’t and that this happens early on.

Certainly my interest was supported by my parents and a variety of teachers, but a lot of it arose from simply reading about Cosmology, or Vulcanism, or Palaeontology. I watched a YouTube of Steven Jay Gould recently saying that when he was a child in the 1950s all children were “in” to Dinosaurs, but that he actually got to make a career out of it. Maybe all children aren’t “in” to dinosaurs in the same way today, perhaps the mystery and sense of excitement has gone.

In the UK at least there appears to be less and less people taking Science and Mathematics. I am not sure what is behind this trend. I read pieces that suggest that Science and Maths are viewed as being “hard” subjects, and people opt for “easier” alternatives. I think creative writing is one of the hardest things to do, so I’m not sure where this perspective comes from.

Perhaps some things that don’t help are the twin images of the Scientist as a white-coated boffin and the Mathematician as a chalk-covered recluse, neither of whom have much of a grasp on the world beyond their narrow discipline. While of course there is a modicum of truth in these stereotypes, they are far from being wholly accurate in my experience.

Perhaps Science has fallen off of the pedestal that it was placed on in the 1950s and 1960s. Interest in Science had been spurred by a range of inventions that had improved people’s lives and often made the inventors a lot of money. Science was seen as the way to a better tomorrow, a view reinforced by such iconic developments as the discovery of the structure of DNA, our ever deepening insight about sub-atomic physics and the unravelling of many mysteries of the Universe. These advances in pure science were supported by feats of scientific / engineering achievement such as the Apollo space programme. The military importance of Science was also put into sharp relief by the Manhattan Project; something that also maybe sowed the seeds for later disenchantment and even fear of the area.

The inevitable fallibility of some Scientists and some scientific projects burst the bubble. High-profile problems included the Thalidomide tragedy and the outcry, however ill-informed, about genetically modified organisms. Also the poster child of the scientific / engineering community was laid low by the Challenger disaster. On top of this, living with the scientifically-created threat of mutually-assured destruction probably began to change the degree of positivity with which people viewed Science and Scientists. People arrived at the realisation that Science cannot address every problem; how much effort has gone into finding a cure for cancer for example?

In addition, in today’s highly technological world, the actual nuts and bolts of how things work are often both hidden and mysterious. While people could relatively easily understand how a steam engine works, how many have any idea about how their iPod functions? Technology has become invisible and almost unimportant, until it stops working.

I am a little wary of Governments fixing issues such as these, which are the result of major generational and cultural trends. Often state action can have unintended and perverse results. Society as a whole goes through cycles and maybe at some future point Science and Mathematics will again be viewed as interesting areas to study; I certainly hope so. Perhaps the current concerns about climate change will inspire a generation of young people to think more about technological ways to address this and interest them in pertinent Sciences such as Meteorology and Climatology.

Ajay-. How would you rate the various tools within the BI industry like in a SWOT analysis (briefly and individually)?

Peter- I am going to offer a Politician’s reply to this. The really important question in BI is not which tool is best, but how to make BI projects successful. While many an unsuccessful BI manager may blame the tool or its vendor, this is not where the real issues lie.

I firmly believe that successful BI rests on four mutually reinforcing pillars:

• understand the questions the business needs to answer,
• understand the data available,
• transform the data to meet the business needs and
• embed the use of BI in the organisation’s culture.

If you get these things right then you can be successful with almost any of the excellent BI tools available in the marketplace. If you get any one of them wrong, then using the paragon of BI tools is not going to offer you salvation.

I think about BI tools in the same way as I do the car market. Not so many years ago there were major differences between manufacturers.

The Japanese offered ultimate reliability, but maybe didn’t often engage the spirit.

The Germans prided themselves on engineering excellence, slanted either in the direction of performance or luxury, but were not quite as dependable as the Japanese.

The Italians offered out-and-out romance and theatre, with mechanical integrity an afterthought.

The French seemed to think that bizarrely shaped cars with wheels as thin as dinner plates were the way forward, but at least they were distinctive.

The Swedes majored on a mixture of safety and aerospace cachet, but sometimes struggled to shift their image of being boring.

The Americans were still in the middle of their love affair with the large and the rugged, at the expense of convenience and value-for-money.

Stereotypically, my fellow-countrymen majored on agricultural charm, or wooden-panelled nostalgia, but struggled with the demands of electronics.

Nowadays, the quality and reliability of cars are much closer to each other. Most manufacturers have products with similar features and performance and economy ratings. If we take financial issues to one side, differences are more likely to related to design, or how people perceive a brand. Today the quality of a Ford is not far behind that of a Toyota. The styling of a Honda can be as dramatic as an Alfa Romeo. Lexus and Audi are playing in areas previously the preserve of BMW and Mercedes and so on.

To me this is also where the market for BI tools is at present. It is relatively mature and the differences between product sets are less than before.

Of course this doesn’t mean that the BI field will not be shaken up by some new technology or approach (in-memory BI or SaaS come to mind). This would be the equivalent of the impact that the first hybrid cars had on the auto market.

However, from the point of view of implementations, most BI tools will do at least an adequate job and picking one should not be your primary concern in a BI project.

Ajay- SAS Institute Chief Marketing Officer, Jim Davis (interviewed with this blog) points to the superiority of business analytics rather than business intelligence as an over hyped term. What numbers, statistics and graphs would you quote rather than semantics to help re direct those perceptions?

I myself use SAS,SPSS, R and find the decision management capabilities as James Taylor calls Decision Management much better enabled than by simple ETL tools or reporting and aggregating graphs tools in many BI tools.

Peter- I have expended quite a lot of energy and hundreds of words on this subject. If people are interested in my views, which are rather different to those of Jim Davis, then I’d suggest that they read them in a series of articles starting with Business Analytics vs Business Intelligence [URL http://peterthomas.wordpress.com/2009/03/28/business-analytics-vs-business-intelligence/ ].

I will however offer some further thoughts and to do this I’ll go back to my car industry analogy. In a world where cars are becoming more and more comparable in terms of their reliability, features, safety and economy, things like styling, brand management and marketing become more and more important.

As the true differences between BI vendors narrow, expect more noise to be made by marketing departments about how different their products are.

I have no problem in acknowledging SAS as a leader in Business Analytics, too many people I respect use their tools for me to think otherwise. However, I think a better marketing strategy for them would be to stick to the many positives of their own products. If they insist on continuing to trash competitors, then it would make sense for them to do this in a way that couldn’t be debunked by a high school student after ten seconds’ reflection.

Ajay- In your opinion what is the average RoI that a small, large medium enterprise gets by investing in a business intelligence platform. What advice would you give to such firms (separately) to help them make their minds?

Peter- The question is pretty much analogous to “What are the benefits of opening an office in China?” the answer is going to depend on what the company does; what their overall strategy is and how a China operation might complement this; whether their products and services are suitable for the Chinese market; how their costs, quality and features compare to local competitors; and whether they have cracked markets closer to home already.

To put things even more prosaically, “How long is a piece of string?”

Taking to one side the size and complexity of an organisation, BI projects come in all shapes and sizes.

Personally I have led Enterprise-wide, all-pervasive BI projects which have had a profound impact on the company. I have also seen well-managed and successful BI projects targeted on a very narrow and specific area.

The former obviously cost more than the latter, but the benefits are commensurately greater. In fact I would argue that the wider a BI project is spread, the greater its payback. Maybe lessons can be learnt and confidence built in an initial implementation to a small group, but to me the real benefit of BI is realised when it touches everything that a company does.

This is not based on a self-interested boosting of BI. To me if what we want to do is take better business decisions, then the greater number of such decisions that are impacted, the better that this is for the organisation.

Also there are some substantial up-front investments required for BI. These would include: building the BI team; establishing the warehouse and a physical architecture on which to deliver your application. If these can be leveraged more widely, then costs come down.

The same point can be made about the intellectual property that a successful BI team develops. This is one reason why I am a fan of the concept of BI Competency Centres [URL http://peterthomas.wordpress.com/2009/05/11/business-intelligence-competency-centres/ ].

I have been lucky enough to contribute to an organisation turning round from losing hundreds of millions of dollars to recording profits of twice that magnitude. When business managers cite BI as a major factor behind such a transformation, then this is clearly a technology that can be used to dramatic effect.

Nevertheless both estimating the potential impact of BI and measuring its actual effectiveness are non-trivial activities. A number of different approaches can be taken, some of which I cover in my article:

Measuring the benefits of Business Intelligence [URL http://peterthomas.wordpress.com/2009/02/26/measuring-the-benefits-of-business-intelligence/ ]. As ever there is no single recipe for success.

Ajay-. Which BI tool/ code are you most comfortable with and what are its salient points?

Peter -Although I have been successful with elements of the IBM-Cognos toolset and think that this has many strong points, not least being relatively user-friendly, I think I’ll go back to my earlier comments about this area being much less important than many others for the success of a BI project.

Ajay -How do you think cloud computing will change BI? What percentage of BI budgets go to data quality and what is eventual impact of data quality on results?

Peter -I think that the jury is still out on cloud computing and BI. By this I do not mean that cloud computing will not have an impact, but rather that it remains unclear what this impact will actually be.

Given the maturity of the market, my suspicion is that the BI equivalent of a Google is not going to emerge from nowhere. There are many excellent BI start-ups in this space and I have been briefed by quite a few of them.

However, I think the future of cloud computing in BI is likely to be determined by how the likes of IBM-Cognos, SAP-BusinessObjects and Oracle-Hyperion embrace the area.

Having said this, one of the interesting things in computing is how easy it is to misjudge the future and perhaps there is a potential titan of cloud BI currently gestating in the garage so beloved of IT mythology.

On data quality, I have never explicitly split out this component of a BI effort. Rather data quality has been an integral part of what we have done. Again I have taken a four-pillared approach:

• improve how the data is entered;
• make sure your interfaces aren’t the problem;
• check how the data has been entered / interfaced;
• and don’t suppress bad data in your BI.

The first pillar consists of improved validation in front-end systems – something that can be facilitated by the BI team providing master data to them – and also a focus on staff training, stressing the importance to the organisation of accurately recording certain data fields.

The second pillar is more to do with the general IT Architecture and how this relates to the Information Architecture, again master data has a role to play, but so does ensuring that the IT culture is one in which different teams collaborate well and are concerned about what happens to data when it leaves “their” systems.

The third pillar is the familiar world of after-the-fact data quality reports and auditing, something that is necessary, but not sufficient, for success in data quality.

Finally there is what I think can be one of the most important pillars; ensuring that the BI system takes a warts-and-all approach to data. This means that bad data is highlighted, rather than being suppressed. In turn this creates pressure for the problems to be addressed where they arise and creates a virtuous circle.

For those who might be interested in this area, I expand on it more in Using BI to drive improvements in data quality [URL http://peterthomas.wordpress.com/2009/02/11/using-bi-to-drive-improvements-in-data-quality/ ].

Ajay- You are well known with England’s rock climbing and boulder climbing community. A fun question- what is the similarity between a BI implementation/project and climbing a big boulder.

Peter -I would have to offer two minor clarifications.

First it is probably my partner who is better known in climbing circles, via here blog [URL http://77jenn.blogspot.com/ ] and articles and reviews that she has written for the climbing press; though I guess I can take credit for most of the photos and videos.

Second, particularly given the fact that a lot of our climbing takes place in Wales, I should acknowledge the broader UK climbing community and also mention our most mountainous region of Scotland.

Despite what many inhabitants of Sheffield might think to the contrary, there is life beyond Stanage Edge [URL http://en.wikipedia.org/wiki/Stanage ].

I have written about the determination and perseverance that are required to get to the top of a boulder, or indeed to the top of any type of climb [URL http://peterthomas.wordpress.com/2009/03/31/perseverance/ ].

I think those same qualities are necessary for any lengthy, complex project. I am a firm believer that the true benefits of BI are only realised when it leads to cultural transformation. Certainly the discipline of change management has many parallels with rock climbing. You need a positive attitude and a strong belief in your ultimate success, despite the inevitable setbacks. If one approach doesn’t yield fruit then you need to either fine-tune or try something radically different.

I suppose a final similarity is the feeling that you get having completed a climb, particularly if it is at the limit of your ability and has taken a long time to achieve. This is one of both elation and deep satisfaction, but is quickly displaced by a desire to find the next challenge.

This is something that I have certainly experienced in business life and I think the feelings will be familiar to many readers.

Biography-

Peter Thomas has led all-pervasive, Business Intelligence and Cultural Transformation projects serving the needs of 500+ users in multiple business units and service departments across 13 European and 5 Latin American countries. He has also developed Business Intelligence strategies for operations spanning four continents. His BI work has won two industry awards including “Best Enterprise BI Implementation”, from Cognos in 2006 and “Best use of IT in Insurance”, from Financial Sector Technology in 2005. Peter speaks about success factors in both Business Intelligence and the associated Change Management at seminars across both Europe and North America and writes about these areas and many other aspects of business, technology and change on his blog [URL http://peterthomas.wordpress.com ].

BI tools are not recommended when they’re the first topic in a BI discussion.

Jill Dyche, Baseline Consulting

Ajay- What approximate Return of Investment would you give to various vendors within Business Intelligence?

Jill- You don’t kid around do you, Ajay? In general the answer has everything to do with the problem BI is solving for a company. For instance, we’re working on deploying operational BI at a retailer right now. This new program is giving people in the stores more power to make decisions about promotions and in-store events. The projected ROI is $300,000 per store per year—and the retailer has over 1000 stores. In another example, we’re working with an HMO client on a master data management project that helps it reconcile patient data across hospitals, clinics, pharmacies, and home health care. The ROI could be life-saving. So, as they say in the Visa commercials: Priceless.

Ajay- What is impact of third party cloud storage and processing do you think will be there on Business Intelligence consulting?

Jill- There’s a lot of buzz about cloud storage for BI, most of it is coming from the VC community at this point, not from our clients. The trouble with that is that BI systems really need control over their storage. There are companies out there—check out a product called RainStor—that do BI storage in the cloud very well, and are optimized for it. But most “cloud” environments geared to BI are really just hosted offerings that provide clients with infrastructure and processing resources that they don’t have in-house.  Where the cloud really has benefits is when it provides significant processing power to companies that can’t build it easily themselves.

Ajay- What are the top writing tips would you give to young struggling business bloggers especially in this recession.

Jill- I’d advise bloggers to write like they talk, a standard admonishment by many a professor of Business Writing. So much of today’s business writing—especially in blogs—is stilted, overly-formal, and pedantic. I don’t care if your grammar is accurate; if your writing sounds like the Monroe Doctrine, no one will read it. (Just give me one quote from the Monroe Doctrine. See what I mean?) Don’t use the word “leverage” when you can use the word “use.” Be genuine and conversational. And avoid clichés like the plague.

Ajay-  How would you convince young people especially women to join more science careers. Describe your own career journey.

Jill- As much as we need those role models in science, high-tech, and math careers, I’d tell them to only embrace it if they really love it. My career path to high-tech was unconventional and unintentional. I started as a technical writer specializing in relational databases just as they were getting hot. One thing I know for sure is if you want to learn about something interesting, be willing to roll up your sleeves and work with it. My technical writing about databases, and then data warehouses, led to some pretty interesting client work.

Sure I’ve coded SQL in my career, and optimized some pretty hairy WHERE clauses. But the bigger issue is applying that work to business problems. Actually I’m grateful that I wasn’t a very good programmer. I’d still be waiting for that infinite loop to finish running.

Ajay- What are the areas within an enterprise where implementation of BI leads to the most gains. And when are BI tools not recommended?

Jill- The best opportunities for BI are for supporting business growth. And that typically means BI used by sales and marketing. Who’s the next customer and what will they buy? It’s answers to questions like these that can set a company apart competitively and contribute to both the top and bottom lines.

Not to be too heretical, but to answer your second question: BI tools are not recommended when they’re the first topic in a BI discussion. We’ve had several “Don’t go into the light” conversations with clients lately where they are prematurely looking at BI tools rather than examining their overall BI readiness. Companies need to be honest about their development processes, existing skill sets, and their data and platform infrastructures before they start phoning up data visualization vendors. Unfortunately, many people engage BI software vendors way before they’re ready.

Ajay- You and your partner Evan wrote what was really the first book on Master Data Management. But you’d been in the BI and data warehousing world before that. Why MDM?

Jill- We just kept watching what our clients couldn’t pull off with their data warehouses. We saw the effort they were going through to enforce business rules through ETL, and what they were trying to do to match records across different source systems. We also saw the amount of manual effort that went into things like handling survivor records, which leads to a series of conversations about data ownership.

Our book (Customer Data Integration: Reaching a Single Version of the Truth, Wiley) has as much to do with data management and data governance as it does with CDI and MDM. As Evan recently said in his presentation at the TDWI MDM Insight event, “You can’t master your data until you manage your data.” We really believe that, and our clients are starting to put it into practice too.

Ajay- Why did you and Evan choose to focus on customer master data (CDI) rather than a more general book on MDM?

Jill- There were two reasons. The first one was because other master data domains like product and location have their own unique sets of definitions and rules. Even though these domains also need MDM, they’re different and the details around implementing them and choosing vendor products to enable them are different. The second reason was that the vast majority of our clients started their MDM programs with customer data. One of Baseline’s longest legacies is enabling the proverbial “360-degree view” of customers. It’s what we knew.

Ajay- What’s surprised you most about your CDI/MDM clients?

Jill- The extent to which they use CDI and MDM as the context for bringing IT and the business closer together. You’d think BI would be ideal for that, and it is. But it’s interesting how MDM lets companies strip back a lot of the tool discussions and just focus on the raw conversations about definitions and rules for business data. Business people get why data is so important, and IT can help guide them in conversations about streamlining data quality and management. Companies like Dell have used MDM for nothing less than business alignment.

Ajay- Any plan to visit India and China for giving lectures?

Jill- I just turned down a trip to China this fall because I had a schedule conflict, which I’m really bummed about. Far as India is concerned, nothing yet but if you’re looking for houseguests let me know.(Ajay- sure I have a big brand new house just ready- and if I visit USA may I be a house guest too?)

About Jill Dyche-

Jill blogs at http://www.jilldyche.com/. where she takes the perpetual challenge of business-IT alignment head on in her trenchant, irreverent style.

Jill Dyché is a partner and co-founder of Baseline Consulting. Her role at Baseline is a combination of best-practice expert, industry gadfly, key client advisor, and all-around thought leader. She is responsible for key client strategies and market analysis in the areas of data governance, business intelligence, master data management, and customer relationship management. Jill counsels boards of directors on the strategic importance of their information investments.

Author

Jill is the author of three books on the business value of IT. Jill’s first book, e-Data (Addison Wesley, 2000) has been published in eight languages. She is a contributor to Impossible Data Warehouse Situations: Solutions from the Experts (Addison Wesley, 2002), and her book, The CRM Handbook (Addison Wesley, 2002), is the bestseller on the topic.

Jill’s work has been featured in major publications such as Computerworld, Information Week, CIO Magazine, the Wall Street Journal, the Chicago Tribune and Newsweek.com. Jill’s latest book, Customer Data Integration (John Wiley and Sons, 2006) was co-authored with Baseline partner Evan Levy, and shows the business breakthroughs achieved with integrated customer data.

Industry Expert

Jill is a featured speaker at industry conferences, university programs, and vendor events. She serves as a judge for several IT best practice awards. She is a member of the Society of Information Management and Women in Technology, a faculty member of TDWI, and serves as a co-chair for the MDM Insight conference. Jill is a columnist for DM Review, and a blogger for BeyeNETWORK and Baseline Consulting.

Alison Bolen  Editor-in-Chief sascom

Here is an interview with Alison Bolen the editor-in-chief of SAScom , online magazaine of the SAS Institute. Alison talks of the challenges in maintaining several of the topmost expertise blogs on SAS ,Business Analytics and Business Intelligence.

Ajay- Describe your career in the technology writing and publishing area. What advice would you give to young Web publishers and content producers just entering the job market in this recession? Describe your journey within SAS.

Alison- I started at SAS in 1999 as a summer student working as a contributing editor for SAS Communications magazine. Before the end of the year, I came on full time and soon transitioned to writing and editing for the Web. At that time, we were just developing the strategy for the customer support site and e-newsletters. As the first editor for the SAS Tech Report, I led marketing efforts that brought in 15,000 opt-in subscribers within six months. A year later, I switched to writing and editing customer success stories, which I enjoyed doing until I took on the role of Editor-in-Chief for sascom® magazine in 2006. We started our blogging program in 2007, and I’ve been actively involved in coaching SAS bloggers for the past two years.

Outside of SAS, I’ve written for Southwest Hydrology Magazine, the Arizona Daily Star and other regional papers. My bachelor’s degree is in magazine journalism and my master’s degree is in technical and business communications.

If you’re just beginning your career as a writer, start a blog and stick with it. There’s no better way to get daily writing practice, learn the basics of search engine optimization and start to understand what works online.

Ajay www.SAS.com/Blogs has many, many blogs by experts, RSS feeds and even covers the annual SAS conference with video content. In terms of social media adaptation, what prompts you to stay ahead of the competition in ensuring marketing and technical communications for brand awareness?

What do you think are the basics of setting up a social media presence for a company, regardless of size?

Alison- Social media excites me because you can cut through the clutter and be real. Our new business forecasting blog by Michael Gilliland is a good example. Teaching people how to forecast better is his top priority, not selling software. Our overarching goal for the blogging program is similar: to share and develop expertise.

We’re big advocates of aligning your social media presence with existing marketing goals. We have a few grass-roots teams interested in social media, and we have a director-level Marketing 2.0 Council that our Social Media Manager Dave Thomas leads to determine broad guidelines and strategies. But the overarching concept is to look at the goals of your individual marketing campaigns first, and then determine which social media channels might help you reach those goals.

Most of all, take off your marketing hat when you enter the blog, network or forum. Social media consists of individuals, for the most part, and not companies, so be sure to offer value as a colleague and build relationships.

Ajay- I noticed that SAS.com/ Blogs are almost ad free – even of SAS products – apart from a simple banner of the company. Was this a deliberate decision, and if so, why?

Alison- Yes, most of the SAS blogs were intentionally created to help establish the individual blogger’s expertise – not to promote SAS products or services. One positive side effect is that SAS – by extension – builds credibility as well. But we really do see the blogs as a place to discuss concepts and ideas more than products and tools.

Ajay- What distinguishes good writers on blogs from bad writers on blogs? How about some tips for technical blog writing and especially editing (since many writers need editors more than they realize)?

Alison- The best blog writers know how to simplify and explain even the most mundane, everyday processes. This is true of personal and technical blog writing. If you can look at your life or your work and see what piece of it others would find interesting or want to know more about – and then know how to describe that sliver of yourself clearly – you have what it takes to be a good blogger. Chris Hemedinger does this well on The SAS Dummy blog.

My biggest editing soapbox right now is to encourage brevity. We’re so used to writing white papers, brochures and magazine articles at SAS that the concept of throwing down 200 words on a random topic from your day is a very foreign exercise. You have to learn how to edit your day – not just your writing – to find those topics and distill those thoughts into quick snippets that keep readers interested. And don’t forget it’s okay to have fun!

Ajay- I balance one blog, small consulting assignments and being a stay-at-home dad for an 18-month old. How easy is it for you to balance being editor of sascom, given the huge content your sites create, and three kids? Does working for SAS and its employee-friendly reputation help you do so?

Alison- I couldn’t balance work and kids without a whole lot of help from friends and family, that’s for sure. And the employee-friendly benefits help too. The biggest benefit is the cultural mindset, though, not any individual policy. My boss and my boss’ boss are both working mothers, and they’re balancing the same types of schedules. There’s an understanding about finding a healthy work-life balance that permeates SAS from top to bottom.

Ajay- As a social media consultant it is a weekly struggle for me to convince companies to discontinue registration for normal content (but keep it for special events), use a lot more video tutorials and share content freely across the Web. Above all, convincing busy senior managers to start writing a blog or an article is an exercise in diplomacy itself. How do you convince senior managers to devote time to content creation?

Alison- In a lot of areas, the content is already being created for analyst presentations, press interviews and consulting briefs. It’s really a matter of understanding how to take those existing materials and re-present them in a more personal voice. Not everyone can – or should – do it. You have to decide if you have the voice for it and whether or not it will bring you value beyond what you’re getting through your existing channels.

Ajay- Any plans to visit India and have a SAS India blogathon?

Alison- Alas, not this year.

Maybe I will visit Cary,NC then

Bio:
Alison Bolen is the Editor of sascom magazine and the sascom voices blog, where SAS experts publish their thoughts on popular and emerging business and technology trends worldwide. Since starting at SAS in 1999, Alison has edited print publications, Web sites, e-newsletters, customer success stories and blogs.

Alison holds a bachelor’s degree in magazine journalism from Ohio University and a master’s degree in technical writing from North Carolina State University.

Working with an R or Scilab on clusters/grids/clouds becomes as simple as working with them locally-

Karim Chine, Biocep.

Ajay- Please describe your career in the field of science. What advice would you give to young science graduates in this recession.

Karim- My original background is in theoretical Physics, I did my Master’s thesis at the Ecole Normale’s Statistical Physics Laboratory where I worked on phase separation in two-dimensional additive mixtures with Dr Werner Krauth. I came to computer science after graduating from the Ecole Polytechnique and I spent two years at TELECOM ParisTech studying software architecture and distributed systems design. I worked then for the IBM Paris Laboratory (VisualAge Pacbase applications’ generator), Schlumberger (Over the Air Platform and Web platform for smartcards personalization services), Air France (SSO deployment) and ILOG (OPL-CPLEX-ODM Development System). This gave me the intense exposure to real world large-scale software design. I crossed the borders of cultural, technical and organizational domains several times and I worked with a broad palette of technologies with some of the best and most innovative engineers. I moved to Cambridge in 2006 and I worked for the European Bioinformatics Institute. It’s where I started dealing with the integration of R into various types of applications. I left the EBI in November 2007. I was looking for an institutional support to help me in bringing into reality a vision that was becoming clearer and clearer about a universal platform for scientific and statistical computing. I failed in getting that support and I have been working on BIOCEP full time for most of the last 18 months without being funded. Few days of consultancy given here and there allowed me to keep going. I spent several weeks at Imperial College, at the National Center for e-Social Sciences and at Berkeley’s department of statistics during that period. Those visits were extremely useful in refining the use cases of my platform. I am still looking for a partner to back the project. You asked me to give advice. The unique advice I would give is to be creative and to try again and again to do what you really want to do. Crisises come and go, they will always do and extreme situations are part of life. I believe hard work and sincerity can prevail anything.

Ajay- Describe BIOCEP’s scope and ambition.

What are the current operational analytics that can be done by users having data.

Karim- My first ambition with BIOCEP is to deliver a universal platform for scientific and statistical computing and to create an open, federative and collaborative environment for the production, sharing and reuse of all the artifacts of computing. My second ambition is to enhance dramatically the accessibility of mathematical and statistical computing, to make HPC a commonplace and to put new analytical, numerical and processing capabilities in the hands of everyone (open science).

The Open source software Conquest has gone very far. Environments like R or Scilab, technologies like Java, Operating Systems like Linux-Ubuntu, and tools like OpenOffice are being used by millions of people. Very little doubt remains about the OSS’s final victory in some domains. The cloud is already a reality and it will take computing to a whole new realm. What is currently missing is the software that, by making the Cloud’s usage seamless, will create new ecosystems and will provide rooms for creativity, innovation and knowledge discovery of an unprecedented scale.

BIOCEP is one more building block into this. BIOCEP is built on top of R and Scilab and anything that you can do within those environments is accessible through BIOCEP. Here is what you have uniquely with this new R/Scilab-based e-platform:

- High productivity via the most advanced cross-platform workbench available for the R environment.

- Advanced Graphics: with BIOCEP, a graphic transducer allows the rendering on client side of graphics produced on server side and enables advanced capabilities like zooming/unzooming/scrolling for R graphics. a client side mouse tracker allows to display dynamically information related to the graphics and depending on coordinates. Several virtual R Devices showing different data can be coupled in zooming/scrolling and this helps comparing visually complex graphics.

- Extensibility with plug-ins: new views (IDE-like views, analytical interfaces…) can be created very easily either programmatically or via drag-and-drop GUI designers.

- Extensibility with server-side extensions: any java code can be packaged and used on server side. The code can interact seamlessly with R and Scilab or provide generic bridges to other software. For example, I provide an extension that allows you to use openoffice as a universal converter between various files formats on server side.

- Seamless High Performance Computing: working with an R or Scilab on clusters/grids/clouds becomes as simple as working with them locally. Distributed computing becomes seamless, creating a large number R and Scilab remote engines and using them to solve large scale problems becomes easier than ever. From the R console the user can create logical links to existing R engines or to newly created ones and use those logical links to pilot the remote workers from within his R session. R functions enable using the logical links to import/export variables from the R session to the different workers and vice versa. R commands/scripts can be executed by the R workers synchronously or asynchronously. Many logical R links can be aggregated into one logical cluster variable that can be used to pilot the R workers in a coordinated way. A cluster.apply function allows the usage of the logical cluster to apply a function to a big data structure by slicing it and sending elementary execution commands to the workers. The workers apply the user’s function to the slices in parallel. The elementary results are aggregated to compose the final result that becomes available within the R session.

- Collaboration: your R/scilab server running in the cloud can be accessed simultaneously by you and your collaborators. Everything gets broadcasted including Graphics. A spreadsheet enables to view and edit data collaboratively. Anyone can write plug-ins to take advantage of the collaborative capabilities of the frameworks. If your IP address is public, you can provide a URL to anyone and get him connect to your locally running R.

- Powerful frameworks for Java developers: BIOCEP provides Frameworks and tools to use R as if it was an Object Oriented Java Toolkit or a Web Toolkit for R-based dynamic application.

- Webservices for C#, Perl, Python users/developers: Most of the capabilities of BIOCEP including piloting of R/Scilab engines on the cloud for distributed computing or for building scalable analytical web application are accessible from most of the programming languages thanks to the SOAP front-end.

- RESTful API: simple URLs can perform computing using R/Scilab engines and return the result as an XML or as graphics in any format. This works like google charts and has all the power of R since the graphic is described with an R script provided as a parameter of the URL. The same API can be exposed on demand by the workbench. This allow for example to integrate a Cloud-R with Excel or OpenOffice. The workbench works as a bridge between the cloud and those applications.

- Advanced Pooling framework for distributed resources: useful for deploying pools of R/scilab engines on multi nodes systems and get them used simultaneously by several distributed client processes in a scalable/optimal way. A supervision GUI is provided for a user friendly management of the pools/nodes/engines.

- Simultaneous use of R and Scilab: Using java scripting, data can be transferred from R to Scilab and vice versa.

Ajay- Could you tell us about a successful BIOCEP installation and what it led to? Can BIOCEP be used by the rest of the R community for other packages? What would be an ideal BIOCEP user /customer for whom cloud based analytics makes more sense ?

Karim- BIOCEP is still in pre-beta stage. However it is a robust and polished pre-Beta that several organizations are already using. Janssen Pharmaceutica is using it to create and deliver statistical applications for drug discovery that use R engines running on their backend servers. The platform is foreseen there as the way to go for an ultimate optimization of some of their data analysis pipelines. Janssen’s head of statistics said to be very much interested in the capabilities given by BIOCEP to statisticians to create their own analytical User Interfaces and deliver them with their models without needing specific software development skills. Shell is creating BIOCEP-based applications prototypes to explore the feasibility and advantages of migrating some of Shell’s applications to the Cloud. One group from Shell Global Solutions is planning to use BIOCEP for running scilab in the cloud for Corrosion simulation modeling. Dr Ivo Dinov’s team at UCLA is studying the migration of some the SOCR applications to the BIOCEP platform as plug-ins and extensions. Dr Ivo Dinov also applied for an important grant for building DISCb (Distributed Infrastructure for Statistical Computing in Biomedicine). If the grant application is successful, BIOCEP will be the backbone at software architecture level of that new infrastructure. In cooperation with the Institute of Biostatistics, Leibniz University of Hannover, Bernd Bischl and Kornelius Rohmeyer have developed a framework to user friendly R-GUIs of different complexity. The toolkit uses BIOCEP as an R-backend since release 2.0. Several small projects have been implemented using this framework and some are in production such as an application for education in biostatistics at the University of Hannover. Also the ESNATS project is planning to use the BIOCEP frameworks. Some development is being done at the EBI to customize the workbench and use it to give to the end user the possibility to run R and Bioconductor on the EBI’s LSF cluster.

I’ve been in touch with Phil Butcher, Sanger’s head of IT and he is considering the deployment of BIOCEP on Sanger’s systems simultaneously with Eucalyptus. The same type of deployment has been discussed with the director of OMII-UK, Neil Chue Hong. BIOCEP’s deployment is probably going to follow the deployment of the Eucalyptus System on NGS. Tena Sakai deployed BIOCEP at the Ernest Gallo Clinic and Research Centre and he is currently exploring the usage of the R on the Cloud via BIOCEP (Eucalyptus / AWS). The platform has been deployed by a small consultancy company specializing in R on several London-based investment banks’ systems. I have had a go ahead form Nancy Wilkins Diher (Director for Science Gateways, SDSC) for deploying on TeraGrid, a deployment on EGEE has been discussed with Dr Steven Newhouse (EGEE Technical Director). Both deployments are in standby at the moment.

Quest Diagnostics is planning to use BIOCEP extensively. Sudeep Talati (University of Manchester) is doing his Master’s project on BIOCEP. He is supervised by Professor Andy Brass and he is exploring the use of a BIOCEP-based infrastructure to deliver microarray analysis workflows in a simple and intuitive way to biologists with and without the Cloud. In Manchester, Robin Pinning (e-Science team leader, Research Computing Services) has the deployment of BIOCEP on Manchester’s research cluster on his agenda…

As I have said, anything that you can do with R including installing, loading and using any R package is accessible through BIOCEP. The platform aims to be universal and to become a tool for productivity and collaboration used by everyone dealing with computing/analytics with or without the cloud.

The Cloud whether it is public or private will be generalized and everyone will become a cloud user in one way or another

Ajay- What motivated you to build BIOCEP and mash cloud computing and R. What scope do you see for cloud computing in developing countries in Asia and Africa?

Karim- When I was at the EBI, I worked on the integration of R within scalable web applications. I explored and tested the available frameworks and tools and all of them were too low level or too simple to answer the problem. I decided to build new frameworks. I had the opportunity to be able to stand on the shoulders of giants.

Simon Urbanek’s packages already bridged the C-API of R with Java reliably. Martin Morgan’s RWebsevices package defined class mappings between R types, including S4 classes, and java.

Progressively R became usable as a Java object oriented toolkit, then as a Java Server. Then I built a pooling framework for distributed resources that made it possible for multiple clients to use multiple R engines optimally.

I started building a GUI to validate the server’s increasingly sophisticated API. That GUI became progressively the workbench.

When I was at Imperial, I worked with the National Grid Service team at the Oxford e-Research Centre to deploy my platform on Oxford’s core cluster. That deployment led to many changes in the architecture to meet all the security requirements.

It was obvious that the next step was to make BIOCEP available on Amazon’s Cloud. Academic Grids are for researchers and the cloud is for everyone. Making the platform work seamlessly on EC2 took few months. With the cloud came the focus on collaborative features (collaborative views, graphics, spreadsheets…).

I can only talk about the example of a country I know, Tunisia, and I guess some of this applies to Asian Countries. Even if the broadband is everywhere today and is becoming accessible and affordable by a majority of Tunisians, I am not sure that the adoption of the cloud would happen soon.

Simple considerations like the obligation to pay for the compute cycles in dollars (and not in dinars) are a barrier for adoption. Spending foreign currencies is subject to several restrictions in general for companies and for individuals; few Tunisians have credit cards that can be used to pay Amazon. Companies would prefer to buy and administer their own machines because the cost of operation and maintenance is lower in Tunisia than it is in Europe/US.

Even if the cloud would help in giving Tunisian researchers access to affordable Computing cycles on demand, it seems that most of them have learned to live without HPC resources and that their research is more theoretical and less computational than it could be. Others are collaborating with research groups in Europe (France) and they are using those European groups’ infrastructures.

Ajay- How would BIOCEP address the problem of data hygiene, data security and privacy. Is encrypted and compressed data transfers supported or planned?

Karim- With BIOCEP, a computational engine is exposed as a distributed component via a single mono-directional HTTP port. When you run such an engine on an EC2 instance you have two options:

• 1/ totally sandbox the machine (via the security group) and leave only the SSH port open.
• Private Key authentication is required to access the machine. In this case you use an SSH Tunnel (created with a tool like Putty for example) which allows you to see the engine as if it was running on your local machine on a port of your choice, the one specified for creating the Tunnel.
• When you start the Virtual Workbench and connect in Http mode to your local host via the specified port, you are effectively connecting to the EC2-R engine. 100% of the information exchanged between your workbench and the engine, including your data, is ciphered thanks to the SSH tunnel.
• The virtual workbench embeds JSCH and can create the Tunnel for you automatically. This mode doesn’t allow collaboration since it requires the private key to let the workbench talk to the EC2 R/Scilab engine.

• 2/ tell the EC2 machine at startup (via the “user data”) to require specific credentials from the user. When the machine starts running, the user needs to provide those credentials to get a session ID and to be able to pilot a virtual EC2 R/Scilab engine. This mode enables collaboration. The client (workbench/scripts) connects to the EC2 machine instance via HTTP (will be HTTPS in a near future).

Ajay- Suppose I have 20 gb per month of data and my organization decided to cut back on the number of annual expensive software. How can the current version of BIOCEP help me do the following?

Karim- Ways BIOCEP can help you right now.

1) Data aggregation and Reporting in terms of spreadsheet, presentation and graphs

• BIOCEP provides a highly programmable server side spreadsheet.

• It can be used interactively as a view of the workbench and simple clicks allow the transfer of data form cells to R variables and vice versa. It can be created and populated from R (console / scripts).

• Any R function can be used within dynamically computed cells. The evaluation of those dynamic cells is done on server side and can use high performance computing functions. Macros allow adding reactivity to the spreadsheets.

• A macro allows the user to execute any R code in response to a value change of an R variable or of the content of a range within a spreadsheet. Variables docking macros allow the mirroring of R variables of any type (vectors, matrixes, data frames..) with ranges within the spreadsheet in Read/Write mode

. Several ready-to-use User Interface components can be created and docked anywhere within the spreadsheet. Those components include

• an R Graphics viewer (PDF viewer) showing Graphics produced by a user-defined R script and reactive on user-defined variables and cell ranges changes,
• customizable sliders mirroring R variables,
• Buttons executing user-defined R code when pressed,
• Combo boxes mirroring factor variables ..

The spreadsheet-based analytical user interface can pilot an R running at any location (local R, Grid R, Cloud R…). It can be created in minutes just by pointing, clicking and copy/pasting.

Cells content+macros+reactive docked components can be saved in a zip file and become a Workbench plug-ins. Like all BIOCEP plug-ins, the spreadsheet-based GUI can be delivered to the end user via a simple URL. It can use a cloud-R or a local R created transparently on the user’s machine.

2) Build time series models, regression models

BIOCEP’s workbench is extensible and I am hoping that contributors will soon start writing plug-ins or converting available GUIs to BIOCEP plug-ins in order to make the creation of those models as easy as possible.

Biography-

Karim Chine
Karim chine graduated from the French Ecole Polytechnique and TELECOM ParisTech. He worked at Ecole Normale Supérieure-LPS (phase separation in two-dimensional additive mixture), IBM (VisualAge Pacbase), Schlumberger (Over the Air Platform and Web platform for smartcards personalization services), Air France (SSO deployment), ILOG (OPL-CPLEX-ODM Development System), European Bioinformatics Institute (Expression Profiler, Biocep) and Imperial College London-Internet Center (Biocep). He contributed to open source software (AdaBroker) and he is the author of the Biocep platform. He currently works on the seamless integration of the new platform within utility computing infrastructures (Amazon EC2), its deployment on Grids (NGS) and its usage as a tool for education and he tries to build collaborations with academic and industrial partners.

You can view his resume here http://www.biocep.net/scan/CV_Karim_Chine_June_2009.pdf

Here is an interview with Gary Cokins , a well respected veteran of the Business Intelligence industry working with the SAS Institute. Gary has just launched his sixth book (wow!) and the gentlemen he is , he agreed to answer these questions en route to his constant traveling.Gary is the expert on performance measurement so we decided to quiz him a bit on this.

CIO’s need to shift their mindset from a technical one to a managerial one.- Gary Cokins, SAS Institute

Ajay -Gary, please describe your career journey from a freshman in college to your position today. What are the key items of advice that you would give to high school students to encourage taking science careers in this recession?

COKINS: I have been very fortunate. After receiving my MBA in 1974 from the Northwestern University Kellogg Graduate School of Management, I worked in industry for ten years. I had the luck of being a financial controller at Fortune 100 corporation division and then becoming operations manager at the same location. I then had to “eat the financial data I was serving,” and it was a true wake-up call – much of the information was at best useless and at worst misleading. Later with Deloitte I was trained on the theory of constraints (TOC) methodology which indicted cost accounting as “enemy number one of productivity.” I learned about the shortcomings with how accountants make assumptions.

In 1988, when Professor Kaplan struck an exclusive relationship with KPMG Peat Marwick, I was recruited to KPMG with about three others with similar operational backgrounds as I to implement activity based cost management (ABC/M) systems but with using an ABC/M modeling software tool. I learned from experience. Four years later, my mentor Bob Bonsack, who had moved on from Deloitte to Electronic Data Systems (EDS) recruited me to head EDS’ cost management consulting. With about fifteen consultants, I was exposed to over a hundred implementations of cost systems. It was there that I experimented with creating a two day “ABC/M rapid prototyping” method that was radically different from the multi-month approach. By starting with a quick vision of what their ABC/M system would look like, companies could iteratively re-model to the level of detail, granularity, and accuracy needed to support analysis and decisions. It did not initially require a huge system, which was why some ABC/M system implementations got into trouble. My major self-realization is that costing is accomplished by modeling cost consumption relationships – an insight that continues to evade many accountants.

When I began to see the application of strategy maps and the balanced scorecard, more light bulbs went off in my brain. I then began truly seeing the organization as a “system” where all the performance improvement methodologies and core processes are inter-connected. I realized that the technologies are no longer the impediment because they are proven. The obstacle is the organization’s thinking – and the mindset of senior management who is presumably doing the leading.

My advice to high school students take your studies more seriously than you even imagine, and spend less time text-messaging everyone you know and focus on the more meaningful relationships. They will eventually be your friends rather than just acquaintances. And take math courses!

Ajay- So what exactly do you do at SAS? And name some interesting anecdotes that led to a lot of value as well as fun for both your company and clients. How does Gary spend his daily day at SAS Institute?

COKINS: My primary role with SAS is to create and deliver thought leadership content about Performance Management leveraging business analytics. I present webinars and write articles, blogs, presentations and also books. For the last four years I have averaged visiting roughly 40 international cities where SAS offices are located to present seminars and meet SAS customers to educate them on the concepts and benefits from Performance Management methodologies.

Recent examples of having fun and providing value to organizations involved providing expert advice to the International Monetary Fund (IMF) in Washington DC and the European Patent Office (EPO) in Brussels. The IMF is at the beginning of implementing an activity based cost management (ABC/M) system whereas the EPO is completing their ABC/M system design. Both organizations were seeking tips for success and pitfalls to avoid. One of my major recommendations was to not under-estimate the natural resistance to change of managers and employees. That is, they need to focus much more on getting their buy-in than worrying if the system is perfect. The value to them is realizing that Performance Management methodologies are much more social than technical.

Regarding my daily activities, when I am not traveling, I am mainly reading articles written by other experts or journalists and then translating my relevant takeaways into content that I can educate others with. I also respond to questions and requests both internally within SAS and externally from customers, management consultants, and university faculty.

Ajay- When you were a young employee, what was the toughest challenge that you faced? What was your worst mistake and how did you overcome it? What lessons did you learn from it?

COKINS: In my first few years in business following my university graduation, my toughest challenge was persuading my supervisors, usually older men than I, to accept my new ideas. I have always been a creative thinker, almost a dreamer; and I was not accustomed to the resistance that managers have to innovations, particularly those suggested by young inexperienced employees fresh from their university schooling.

My worst mistake was developing a computer program that automatically suggested treasury cash balance transfers to optimize the corporate cash management system of my first employer, a large Fortune 100 corporation. My computer program was basically replacing the decisions made by the corporate cash manager and part of his job. I overcame this disappointment by learning what needs the corporate cash manager did have and developing a different computer program that solved his needs. With its success, he eventually accepted the first computer program.

My lesson was one should first understand what people may want rather than trying to impose on them what you think they need without involving them.

Ajay- Looking back on your distinguished career, what project are you proud of the most? What project would you do over again if given the chance?

COKINS: In 1973 I became a financial controller of a large division of another Fortune 100 manufacturer. I created a rolling financial planning and forecast software program, using pre-spreadsheet software from a mainframe (years before personal computers and Excel). The program modeled product line sales forecasts by month and integrated both the income statement and balance sheet. It became a valuable tool for the executive team to suggest and immediately see varying sales levels as a “what if” scenario builder to calculate the different profit and working capital results. The executive team marveled at how analytical software, in contrast to our transactional ERP-like system, could make sense of the complexity of our operations with thousands of products and customers.

Regarding a project that fell short of expectations, I actually did get a chance to do it over again. As a consultant with Deloitte, I lead a project designing and implementing an activity based cost management (ABC/M) system using the client’s general ledger accounting software. It took many months, and when finished it was too complex for the client to fully understand. Several years later with a similar project I applied a rapid prototyping with iterative re-modeling approach that involved the company’s managers from the first day. (I mentioned this approach in my reply to the first question.) We completed the ABC/M system in just a few weeks, and everyone understood it and also how to interpret the information for analysis and decisions. I have since been a proponent of this type of rapid learning and system design approach.

Ajay- What do people do for fun at SAS Institute do when not creating or selling algorithms? How is SAS reaching out to other members of the analytics community in terms of basic science and development?

COKINS: SAS employees are inspired by our CEO, Dr. Jim Goodnight, who founded SAS roughly 35 years ago. Dr. Goodnight loves solving problems of all flavors. For fun, but also part of our jobs, SAS employees search for problems that only computer software can solve.

SAS’ offerings evolve by listening to our customers, who are typically scientists, researchers, and business analysts. Drug development and marketing analysts are examples. Our customers are our “community.” We motivate them, with formal methods of collecting input from them, to share with us enhancements to our future versions of our software.

Ajay- Describe your new book on Performance Management from the point of a beginner. Assume that I am a college student who does not know why I should read it. Then assume that I am a CIO and have little time to read it. What is in it for a CIO?

COKINS: This is my sixth book I have written. My first four books were about activity based cost management (ABC/M) and the last two about Performance Management. What is different about this second book is it immediately clarifies the confusion and ambiguity about what Performance Management is and is not. It is also written in a humorous and simplified way with lots of analogies and metaphors, such as all of the Performance Management methodologies integrated together like gears in an automobile engine and with a GPS for predictive navigation and dashboards for feedback. Beginners perceive each methodology, such as a balanced scorecard or customer relationship management system, are stand-alone tools. There is synergy when they are integrated.

CIOs have similar needs. They need to shift their mindset from a technical one to a managerial one. Just a few chapters from this book can help CIOs see the broad picture of how all of their organizations processes fit together, and how they can be aligned to efficiently execute the ever-adjusting strategy that the executives continuously formulate with operations.

Biography and Contact Information

Gary Cokins, CPIM

(gary.cokins@sas.com; phone 919 531 2012)

http://blogs.sas.com/cokins

Gary Cokins is a global product marketing manager involved with performance management solutions with SAS, a leading provider of performance management and business analytics software headquartered in Cary, North Carolina. Gary is an internationally recognized expert, speaker, and author in advanced cost management and performance improvement systems. Gary received a BS degree with honors in Industrial Engineering/Operations Research from Cornell University in 1971. He received his MBA from Northwestern University’s Kellogg School of Management in 1974.

Gary began his career as a strategic planner FMC’s Link-Belt Division and then served as Financial Controller and Operations Manager. In 1981 Gary began his management consulting career first with Deloitte Consulting. Next with KPMG Peat Marwick, Gary was trained on ABC by Harvard Business School Professors Robert S. Kaplan and Robin Cooper. More recently, Gary headed the National Cost Management Consulting Services for Electronic Data Systems (EDS)/ A.T. Kearney.

Gary was the lead author of the acclaimed An ABC Manager’s Primer (ISBN 0-86641-220-4) sponsored by the Institute of Management Accountants (IMA). Gary’s second book, Activity Based Cost Management: Making it Work (ISBN 0-7863-0740-4), was judged by the Harvard Business School Press as “read this book first.” A reviewer for Gary’s third book, Activity Based Cost Management: An Executive’s Guide (ISBN 0-471-44328-X) said, Gary has the gift to take the concept that many view as complex and reduce it to its simplest terms.” This book was ranked number one in sales volume of 151 similar books on BarnesandNoble.com. Gary has also written Activity Based Cost Management in Government (ISBN 1-056726-110-8). His latest books are Performance Management: Finding the Missing Pieces to Close the Intelligence Gap (ISBN 0-471-57690-5) and Performance Management: Integrating Strategy Execution, Methodologies, Risk, and Analytics (ISBN 978-0-470-44998-1).

Mr. Cokins participates and serves on committees including: CAM-I, the Supply Chain Council, the International Federation of Accountants (IFAC), and the Institute of Management Accountants. Mr. Cokins is a member of Journal of Cost Management Editorial Advisory Board. Cokins can be reached at gary.cokins@sas.com . His blog is at http//:blogs.sas.com/cokins

and his latest book can also be previewed at http://www.sas.com/apps/pubscat/bookdetails.jsp?catid=1&pc=62401

Date    Name of Interviewee    Designation and Organization

09-Jun    Karl Rexer                          President, Rexer Analytics
05-Jun    Jim Daves                          CMO, SAS Institute
04-Jun    Paul van Eikeren                 President and CEO, Blue Reference
29-May    David Smith                      Director of Community, REvolution Computing
17-May    Dominic Pouzin                 CEO, Data Applied
11-May    Bruno Delahaye                 VP, KXEN
04-May    Ron Ramos                        Director, Zementis
30-Apr    Oliver Jouve                       VP, SPSS Inc
21-Apr    Fabian Dill                         Co- Founder, Knime.com
18-Apr    Alicia Mcgreevey                 Head Marketing, Visual Numerics
27-Mar    Francoise Soulie Fogelman    VP, KXEN
17-Mar    Jon Peck                            Principal Software Engineer, SPSS Inc
06-Mar    Anne Milley                        Director of product marketing, SAS Institute
04-Mar    Anne Milley                        Director of product marketing, SAS Institute
03-Feb    Phil Rack                            Creator, Bridge to R,and CEO Minequest
03-Feb    Michael Zeller                     CEO, Zementis
31-Jan    Richard Schultz                   CEO, Revolution Computing
21-Jan    Bob Muenchen                    Author, R for SAS and SPSS Users
13-Jan    Dr Graham Williams           Creator, Rattle GUI for R
05-Jan    Roger Haddad                    CEO, KXEN
26-Sep    June Dershewitz                  VP, Semphonic
04-Sep    Vincent Granville                 Head, Analyticbridge

The URl’s to specific interviews are also in this sheet.

http://spreadsheets.google.com/pub?key=rWTqcMe9mqwHeFv1e4GS_yg&single=true&gid=0&range=a1%3Ae24&output=html

Almost 20% of data miners report that their company/organizations have only minimal analytic capabilities – Karl Rexer

Ajay- Describe your career in science. What advice would you give to young science graduates in this recession? What advice would you give to high school students choosing from science – non science careers?

Karl- My interests in science began as a child. My father has multiple science degrees, and I grew up listening to his descriptions of the cool things he was building, or the cool investigative tools he was using, in his lab. He worked in an industrial setting, so visiting was difficult. But when I could, I loved going in to see the high-temperature furnaces he was designing, the carbon-fiber production processes he was developing, and the electron microscope that allowed him to look at his samples. Both of my parents encouraged me to ask why, and to think critically about both scientific and social issues. It was also the time of the Apollo moon landings, and I was totally absorbed in watching and thinking about them. Together these things motivated me and shaped my world-view.

I have also had the good fortune to work across many diverse areas and with some truly outstanding people. In graduate school I focused on applied statistics and the use of scientific methods in the social sciences. As a grad student and young academic, I applied those skills to researching how our brains process language. But on the side, I pursued a passion for using the scientific method and analytics to address ….well anything I could. We called it “statistical consulting” then, but it often extended to research design and many other parts of the scientific process. Some early projects included assisting people with AIDS outcome studies, psycholinguistic research, and studies of adolescent adjustment.

My first taste of applying these skills outside of an academic environment was with my mentor Len Katz. The US Navy hired us to help assess the new recruits that were entering the submarine school. Early identification of sailors who would excel in this unusual and stressful environment was critical. Perhaps even more important was identifying sailors who would not perform well in that environment. Luckily, the Navy had years of academic and psychological testing on many sailors, and this data proved quite useful in predicting later job performance onboard the submarines. Even though we never got the promised submarine ride, I was hooked on applying measurement, scientific methods, and analytics in non-academic settings.

And that’s basically what I have continued to do – apply those skills and methods in diverse scientific and business settings. I worked for two banks and two consulting firms before founding Rexer Analytics in 2002. Last year we supported 30 clients. I’ve got great staff and they have great quant skills. Importantly, we also don’t hesitate to challenge each other, and we’re continually learning from each other and from each client engagement. We share a love of project diversity, and we seek it out in our engagements. We’ve forecasted sales for medical devices, measured B2B customer loyalty, identified manufacturing problems by analyzing product returns, predicted which customers will close their bank accounts, analyzed millions of tax returns, helped identify the dimensions of business team cohesion that result in better performance, found millions of dollars of B2B and B2C fraud, and helped many companies understand their customers better with segmentations, surveys, and analyses of sales and customer behavior.

The advice I would give to young science grads in this recession is to expand your view of where you can apply your scientific training. This applies to high school students considering science careers too. All science does not happen in universities, labs and other traditional science locations. Think about applying scientific methods everywhere! Sometimes our projects at Rexer Analytics seem far away from what most people would consider “science.” But we’re always asking “what data is available that can be brought to bear on the business issue we’re addressing.” Sometimes the best solution is to go out and collect more data – so we frequently help our clients improve their measurement processes or design surveys to collect the necessary data. I think there are enormous opportunities for science grads to apply their scientific training in the business world. The opportunities are not limited to physics wiz-kids making models for Wall Street trading or computer science students moving to Silicon Valley. One of the best analytic teams I ever worked on was at Fleet Bank in the late 90s. We had an economist, two physicists, a sociologist, a psychologist, an operations research guy, and person with a degree in marketing science. We were all very focused on data, measurement, and analytic methods.

I recommend that all science grads read Tom Davenport’s book Competing on Analytics *. It illustrates, with compelling examples, how businesses can benefit from using science and analytics. Several examples in Tom’s book come from Gary Loveman, CEO of Harrah’s Entertainment. I think that Gary also serves as a great example of how scientific methods can be applied in every industry. Gary has a PhD in economics from MIT, he’s worked at the Federal Reserve Bank, he’s been a professor at Harvard, but more recently he runs the world’s largest casino and gaming company. And he’s famously said many times that there are three ways to get fired at Harrah’s: steal, harass women, or not use a control group. Business leaders across all industries are increasingly wanting data, analytics and scientific decision-making. Science grads have great training that enables them to take on these roles and to demonstrate the success of these methods.

Ajay- One more survey- How does the Rexer survey differentiate itself from other surveys out there?

Karl- The Annual Rexer Analytics Data Miner Survey is the only broad-reaching research that investigates the analytic behaviors, views and preferences of data mining professionals. Each year our sample grows — in 2009 we had over 700 people around the globe complete our survey. Our participants include large numbers of both academic and business people.

Another way our survey is differentiated from other surveys is that each year we ask our participants to provide suggestions on ways to improve the survey. Incorporating participants’ suggestions improves our survey. For example, in 2008 several people suggested adding questions about model deployment and off-shoring. We asked about both of these topics in the 2009 survey.

Ajay -Could you please share some sneak previews of the survey results? What impact is the recession likely to have on IT spending?

Karl- We’re just starting to analyze the 2009 survey data. But, yes, here’s a peek at some of the findings that relate to the impact of the recession:

* Many data miners report that funding for data mining projects can sometimes be a problem.
* However, when asked what will happen in 2009 if the economic downturn continues, many data miners still anticipate that their company/organization will conduct more data mining projects in 2009 than in previous years (41% anticipate more projects in 2009; 27% anticipate fewer projects).
* The vast majority of companies conduct their data mining internally, and very few are sending data mining off-shore.

I don’t have a crystal ball that tells me about the trends in overall corporate spending on IT, Business Intelligence, or Data Mining. It’s my personal experience that many budgets are tight this year, but that key projects are still getting funded. And it is my strong opinion that in the coming years many companies will increase their focus on analytics, and I think that increasingly analytics will be a source of competitive advantage for these companies.

There are other people and other surveys that provide better insight into the trends in IT spending. For example, Gartner’s recent survey of over 1,500 CIOs (http://www.gartner.com/it/page.jsp?id=855612 ) suggests that 2009 IT spending is likely to be flat. I’m personally happy to see that in the Gartner survey, Business Intelligence is again CIOs’ top technology priority, and that “increasing the use of information/analytics” is the #5 business priority.

Ajay- I noticed you advise SPSS among others. Describe what an advisory role is for an analytics company and how can small open source companies get renowned advisors?

Karl- We have advised Oracle, SPSS, Hewlett-Packard and several smaller companies. We find that advisory roles vary greatly. The biggest source of variation is what the company wants advice about. Example include:

* assessing opportunity areas for the application of analytics
* strategic data assessments
* analytic strategy
* product strategy
* reviewing software

Both large and small companies that look to apply analytics to their businesses can benefit from analytic advisors. So can open source companies that sell analytic software. Companies can find analytic advisors in several ways. One way is to look around for analytic experts whose advice you trust, and hire them. Networking in your own industry and in the analytic communities can identify potential advisors. Don’t forget to look in both academia and the business world. Many skilled people cross back and forth between these two worlds. Another way for these companies to obtain analytic advice is to look in their business networks and user communities for analytic specialists who share some of the goals of the company – they will be motivated for your company to succeed. Especially if focused topic areas or time-constrained tasks can be identified, outside experts may be willing to donate their time, and they may be flattered that you asked.

Ajay- What made you decide to begin the Rexer Surveys? Describe some results of last year’s surveys and any trends from the last three years that you have seen.

Karl- I’ve been involved on the organizing committees of several data mining workshops and conferences. At these conferences I talk with a lot of data miners and companies involved in data mining. I found that many people were interested in hearing about what other data miners were doing: what algorithms, what types of data, what challenges were being faced, what they liked and disliked about their data mining tools, etc. Since we conduct online surveys for several of our clients, and my network of data miners is pretty large, I realized that we could easily do a survey of data miners, and share the results with the data mining community. In the first year, 314 data miners participated, and it’s just grown from there. In 2009 over 700 people completed the survey. The interest we’ve seen in our research summaries has also been astounding – we’ve had thousands of requests. Overall, this just confirms what we originally thought: people are hungry for information about data mining.

Here is a preview of findings from the initial analyses of the 2009 survey data:

* Each year we’ve seen that the most commonly used algorithms are decision trees, regression, and cluster analysis.
* Consistently, some of the top challenges data miners report are dirty data and explaining data mining to others. Previously, data access issues were also reported as a big challenge, but in 2009 fewer data miners reported facing this challenge.
* The most prevalent concerns with how data mining is being utilized are: insufficient training of some data miners, and resistance to using data mining in contexts where it would be beneficial.
* Data mining is playing an important role in organizations. Half of data miners indicate their results are helping to drive strategic decisions and operational processes.
* But there’s room for data mining to grow – almost 20% of data miners report that their company/organizations have only minimal analytic capabilities.

Bio-

Karl Rexer, PhD is President of Rexer Analytics, a small Boston-based consulting firm. Rexer Analytics provides analytic and CRM consulting to help clients use their data to make better strategic and tactical decisions. Recent projects include fraud detection, sales forecasting, customer segmentation, loyalty analyses, predictive modeling for cross-sell and attrition, and survey research. Rexer Analytics also conducts an annual survey of data miners and freely distributes research summaries to the data mining community. Karl has been on the organizing committees of several international data mining conferences, including 3 KDD conferences, and BIWA-2008. Karl is on the SPSS Customer Advisory Board and on the Board of Directors of the Oracle Business Intelligence, Warehousing, & Analytics (BIWA) Special Interest Group. Karl and other Rexer Analytics staff are frequent invited speakers at MBA data mining classes and conferences.

To know more do check out the website on www.rexeranalytics.com

*

Traditional business intelligence (BI) as we know it is outdated and insufficient-

http://decisionstats.posterous.com/decisionstats-interview-paul-van-eikeren-infe

Interview Paul van Eikeren Inference for R

Here is an interview with Paul van Eikeren, President and CEO of Blue Reference, Inc. Paul heads up a startup company addressing the need of information workers to have easier-cheaper-faster access to high-end data mining, analysis and reporting capabilities from software like R, S-plus, MATLAB, SAS, SPSS, python and ruby. His recent product Inference for R has been causing waves within the analytical fraternity across both R users and SAS users, especially given the fact that it is quite well designed, has a great GUI, and is priced rather reasonably.

A few weeks ago, rumour had it the SAS Institute was reportedly buying out the Inference for R product ( Note the merger and acquisition question below)

Rather curious to know about this company, I happened to met Ben Hincliffe at the www.analyticbridge.com site which with 5000 members has the largest number of data analytics and many business intelligence members as well). Ben who recently authored a guest post for Sandro at Data Mining Blog then put across my request to interview with Paul, the CEO for Blue Reference. Existing products for Blue Reference include additional analytical packages like Inference for Matlab etc.

Paul is an extremely seasoned person with years in the analytical fraternity and with a Phd from MIT. Here is Paul’s vision on his company and analytics product development.
pve1

Ajay: Describe your career journeys. What advice would you give to today’s young people of following careers in science.

Paul: I have been blessed with extremely productive and diversified career journey. After receiving undergraduate and graduate degrees in chemistry, I taught chemistry and carried out research as a college professor for 14 years. During the next 12 years I spend heading R&D teams at three different startup companies focused on the application of novel processing technology for use in drug discovery and development. And using that wealth of acquired experience, I have had the good fortune to successfully co-found and develop with my son Josh, two startup companies (IntelliChem and Blue Reference) directed at the use of informatics to drive more efficient and effective Research, Development, Manufacturing and Operations.

In my journey I have had the opportunity to counsel many young people regarding their career choices. I have offered two principal pieces of advice: one, for the right person, science represents an outstanding opportunity for a productive and satisfying career; and two, a science education provides an outstanding stepping stone to careers in other fields. A study disclosed in a recent Wall Street Journal article (Sarah E. Needleman, “Doing the Math to Find the Good Jobs, 26 January 2009) revealed that mathematicians land the top spot in the new rankings of the best occupations. Science-linked occupations took 7 out of the top 20 spots.

These ratings suggest that the problem solving and innovation aspects of scientific occupations are much less stressful than other occupations, which leads to high job satisfaction. But does one have to be a genius to have a successful career in science? An interesting read on this subject is the book by Robert Weisberg (Creativity: Beyond the Myth of the Genius) in which he dispels the myth of the genius being the results of a genetic gift. Weisberg argues, convincingly, that a genius exhibits three elements: (1) a basic intellectual capacity; (2) a high level of motivation/determination, which enables the genius to remain focused; and (3) immersion in their chosen field, typically represented by over 10,000 hours of study/practice/experience. It turns out that the latter element is the principal differentiator, and fortunately, it is something one has control over.

Ajay: Describe the journey that Blue Reference has made leading to its current product line, including Inference for R.

Paul: The Inference product suite represents a natural extension beyond the Electronic Laboratory Notebook (ELN) product we developed at our previous company, IntelliChem. ELNs are used by scientists and technicians to document research, experiments and procedures performed in a laboratory. The ELN is a fully electronic replacement of the paper notebook. IntelliChem (sold to Symyx in 2004) was a leader in deployment of ELNs at global pharmaceutical companies.

After seeing the successful adoption of ELNs in the laboratory, we saw an opportunity to improve upon the utility of ELN documents and the data contained therein. Essentially, we developed Inference to be a platform for enabling MS Office documents with powerful, flexible, and transparent analytic capabilities – what we call “dynamic documents” or “document mashups”. Executable code from high-level scripting languages like R, MATLAB, and .NET, is combined with data and explanatory text in the document canvas to transform it from a static record into an analytic application.

The pharmaceutical industry, in cooperation with the FDA, has begun to look at ways to implement quality by design (QbD) practices as an alternative to quality by end-testing. QbD comprises a systematic application of predictive analytics to the drug R&D process such that development timelines and costs are reduced while drug safety and efficacy is improved.

Statistical modeling and analysis plays a key role in QbD as a tool for identifying critical quality attributes and confining their variability to a specified design space. Dynamic documents fit nicely into this paradigm, and we’re currently using Inference as a platform to develop an enterprise solution for QbD. You can visit www.InferenceForQbD.com for more information about our QbD product.

Along the way, we recognized the need for Inference outside of the pharmaceutical industry. The Inference for R, Inference for MATLAB, and Inference for.NET versions are meant to serve users of these technical computing languages who have analysis, publishing, reporting, collaboration, and reproducible research needs that are best served by a document centric environment. By using Microsoft Word, Excel and PowerPoint as the “front end,” we can serve the the 500 million users that use Microsoft Office as their principal desktop productive application.

Ajay: What is the pricing strategy for Inference for Matlab and Inference for R – and how do you see the current recession as an opportunity for analytical products.

Paul: Our strategy is to reach out to the market Microsoft Office users that would benefit from easy access to datamining and predictive analytics capabilities within their principal desktop productivity tool. Accordingly, we have offered the Inference product at the low price of $199 for a single user/one year subscription. Additionally, because it is implemented on top of an existing installation of Microsoft Office, the cost of training, support and maintenance are expected to be minimal.

create-a-simple-user-interface-for-your-r-application
create-a-simple-user-interface-for-your-r-application

r-code-directly-in-excel-to-customize-your-analysis
r-code-directly-in-excel-to-customize-your-analysis

graphical-output-in-an-excel-tab
graphical-output-in-an-excel-tab

Ajay: Your product seems to follow a nice fit where both open source as well as proprietary packages from Microsoft( .Net) are working together to give the customer a nice solution. Do you believe it is possible that big companies and big open source communities can work together to create some software rather than just be at loggerheads.

Paul: Absolutely. We’re seeing momentum build for open source analytic solutions as the economy impacts companies, both small and large. We saw this take place in the back office with implementation of Linux and Apache Web servers, and now we’re starting to see it in the front office. Smart IT teams are looking for creative ways to stretch their resources, forcing them to look beyond established, but expensive, software products.

We’ve encountered concrete evidence of this in the financial industry. Fresh on the heels of the credit crisis, investment banks and hedge funds have begun to realize that their risk models and supporting software infrastructure are inadequate. In response, quantitative finance and risk analysts are increasingly turning to the open source R statistical computing environment for improved predictive analytics.

R has a core group of devotees in academia that drive innovation, making it a comprehensive venue for development of leading-edge data analysis methods. In order to leverage these tools, banks need a way

for R to play nicely with their existing personnel and IT infrastructure. This is where Inference for R produces real value. It transforms MS Office into platform for the development, distribution, and maintenance of R based quantitative tools – enabling production level predictive analytics.

Commercial distributions of R address issues of scalability and support, which might otherwise be subjects of concern. For example, REvolution Computing distributes an optimized, validated and supported distribution of R, providing peace of mind to corporate IT. REvolution also offers Enterprise R, a distribution of R for 64-bit, high performance computing.

Ajay: Please name any successful customer testimonials for Inference for R.

Paul: We have been working with the director of quantitative analytics at a large international bank. He reported that he has successfully distributed R applications to his team of research analysts and portfolio managers based on Inference in Excel. Use of this strategy eliminated the need to code complex models in Visual Basic for applications, which is time consuming and error prone.

Ajay: Also are there any issues with licensing and IP for mixing open source code and proprietary code.

Paul- The licensing issues with open source R pertain to distributing R. There are no licensing restrictions in using R. Accordingly, we do not distribute R. Rather, our customers install R separately and Inference recognizes the installation.

Ajay: So R is free and I can get Open Office for free. What are the five specific uses where Inference for R can score an edge over this and make me pay for the solution.

Paul: R is free, and many R enthusiasts would argue that all you need for R is a Linux operating system like Ubuntu, a text editor such as Emacs, and R’s command line interface. For some highly-skilled R users this is sufficient; for the new and average R user this is a nightmare.

Many people think that the largest fraction of the cost of implementing new software is the cost of the license. In actuality, and especially in the corporate world, it is the cost of training, user support, software maintenance, and the costs of switching the user base to the new software. Free open source software does not help here. Hence there is a strong ROI argument to be made to build new software application on top of existing systems that have worked well.

Additionally, successful implementation of open source software like R requires a baseline of integration with existing systems. The fact is that Microsoft operating systems dominate the business world, as does Microsoft Office. If one is serious about using R to address the analytic needs of big business, tight integration with these systems is imperative.

Ajay: Any plans for a web hosted SaaS version for Inference for R soon?

Paul: The natural progression of Inference for R to SaaS will coincide with the next release of Office (Office 2010 or Office 14), which we expect to be largely SaaS enabled.

Ajay: Name some alliances and close partners working with Blue Reference

- and what we can expect from you in terms of product launches in 2009.

Paul: We have created a product development consortium in partnership involving ‘top ten’ global pharmaceutical companies The consortium is guiding the development of an enterprise solution for Quality by Design (QbD), using Inference for R as the platform.

We are working with several consulting firms specializing in IT solutions for specialized markets like risk management and predictive analytics.

We are also working with several technology partners who have complementary products and where integration of their products with Inference provides clear and significant value to customers.

Ajay: Any truth to the rumors of an acquisition by a BIG analytics company?

Paul: Our business strategy is centered on growth through partnerships with others. Acquisition is one means to execute that strategy.

Ajay: How do you see this particular product (for R) shaping up down the years.

Paul: R’s success can be attributed, in large part, to the support of its loyal open source community. Its enthusiastic use in academia bodes very well for its growth as a cutting-edge analytics tool. It is just a matter of time before commercial analytic solutions powered by R become de rigueur. We’re happy to be at the tip of the spear.

Ajay: Any Asia plans for Blue Reference or are you still happy with the Oregon location. How do you plan to interact with graduate schools and academia for your products.

Paul: Although we don’t have a major private university in our backyard, Oregon State University has opened a campus here. And, we’ve been in dialogue with the global Academic community from day one. Over 100 academic institutions around the world use Inference through our academic licensing program. Inference is a great tool for preparing dynamic lessons and publishing reproducible research.

Our Central Oregon location is home to a growing high-tech sector that we’ve been a part of for decades. We’ve had success building large and profitable companies here. Bend attracts Silicon Valley types who come here for vacation and don’t want to leave – they just can’t seem to resist the quality of life and bountiful recreational opportunities that this area offers. It’s a good mix of work and play.

Biography

Paul van Eikeren is President and CEO of Blue Reference, Inc. He is responsible for guiding the strategic direction of the company through novel products and services development, partnerships and alliances in the realm of application of informatics to faster-cheaper-better research, development, manufacturing and operations. Van Eikeren is a successful serial entrepreneur, which includes the co-founding of IntelliChem with his son Josh and its ultimate sale to Symyx Technologies. He has headed up R&D at several startup companies focused on drug discovery and development including Sepracor Inc., Argonaut Technologies, Inc, and Bend Research, Inc. He served as Professor of Chemistry and Biochemistry at Harvey Mudd College of Science and Engineering. He is author/co-author and inventor/co-inventor in over 50 scientific articles and patents directed at the application of chemical, biochemical and computational technologies. Van Eikeren holds a BA degree in Chemistry from Columbia University and a PhD in Chemistry from MIT.bluereference-logo

Ajay- To know more I recommend checking out the free evaluation at http://inferenceforr.com/ especially if you need to rev up your MS office Installation with greater graphics and analytics juice.

” Our development team spent more than six months making R work on 64-bit Windows (and optimizing it for speed), which we released as REvolution R Enterprise bundled with ParallelR.” David Smith –

Ajay -Tell us about your journey in science. In particular tell us what attracted you to R and the open source movement.

David- I got my start in science in 1990 working with CSIRO (the government science organization in Australia) after I completed my degree in mathematics and computer science. Seeing the diversity of projects the statisticians there worked on really opened my eyes to statistics as the way of objectively answering questions about science.

That’s also when I was first introduced to the S language, the forerunner of R. I was hooked immediately; it was just so natural for doing the work I had to do. I also had the benefit of a wonderful mentor, Professor Bill Venables, who at the time was teaching S to CSIRO scientists at remote stations around Australia. He brought me along on his travels as an assistant. I learned a lot about the practice of statistical computing helping those scientists solve their problems (and got to visit some great parts of Australia, too).

Ajay- How do you think we should help bring more students to the fields of mathematics and science-

David- For me, statistics is the practical application of mathematics to the real world of messy data, complex problems and difficult conclusions. And in recent years, lots of statistical problems have broken out of geeky science applications to become truly mainstream, even sexy. In our new information society, graduating statisticians have a bright future ahead of them which I think will inevitably draw more students to the field.

Ajay- Your blog at REVolution Computing is one of the best technical corporate blogs. In particular the monthly round up of new packages, R events and product launches all written in a lucid style. Are there any plans for a REvolution computing community or network as well instead of just the blog.

David- Yes, definitely. We recently hired Danese Cooper as our Open Source Diva to help us in this area. Danese has a wealth of experience building open-source communities, such as for Java at Sun. We’ll be announcing some new community initiatives this summer. In the meantime, of course, we’ll continue with the Revolutions blog, which has proven to be a great vehicle for getting the word out about R to a community that hasn’t heard about it before. Thanks for the kind words about the blog, by the way — it’s been a lot of fun to write. It will be a continuing part of our community strategy, and I even plan to expand the roster of authors in the future, too. (If you’re an aspiring R blogger, please get in touch!)

Ajay- I kind of get confused between what exactly is 32 bit or 64 bit computing in terms of hardware and software. What is the deal there. How do Enterprise solutions from REvolution take care of the 64 bit computing. How exactly does Parallel computing and optimized math libraries in REvolution R help as compared to other flavors of R.

David- Fundamentally, 64-bit systems allow you to process larger data sets with R — as long as you have a version of R compiled to take advantage of the increased memory available. (I wrote about some of the technical details behind this recently on the blog.)  One of the really exciting trends I’ve noticed over the past 6 months is that R is being applied to larger and more complex problems in areas like predictive analytics and social networking data, so being able to process the largest data sets is key.

One common mis perception is that 64-bit systems are inherently faster than their 32-bit equivalents, but this isn’t generally the case. To speed up large problems, the best approach is to break the problem down into smaller components and run them in parallel on multiple machines. We created the ParallelR suite of packages to make it easy to break down such problems in R and run them on a multiprocessor workstation, a local cluster or grid, or even cloud computing systems like Amazon’s EC2 .

” While the core R team produces versions of R for 64-bit Linux systems, they don’t make one for Windows. Our development team spent more than six months making R work on 64-bit Windows (and optimizing it for speed), which we released as REvolution R Enterprise bundled with ParallelR. We’re excited by the scale of the applications our subscribers are already tackling with a combination of 64-bit and parallel computing”

Ajay-  Command line is oh so commanding. Please describe any plans to support or help any R GUI like rattle or R Commander. Do you think Revolution R can get more users if it does help a GUI.

David- Right now we’re focusing on making R easier to use for programmers by creating a new GUI for programming and debugging R code. We heard feedback from some clients who were concerned about training their programmers in R without a modern development environment available. So we’re addressing that by improving R to make the “standard” features programmers expect (like step debugging and variable inspection) work in R and integrating it with the standard environment for programmers on Windows, Visual Studio.

In my opinion R’s strength lies in its combination of high-quality of statistical algorithms with a language ideal for applying them, so “hiding” the language behind a general-purpose GUI negates that strength a bit, I think. On the other hand it would be nice to have an open-source “user-friendly” tool for desktop statistical analysis, so I’m glad others are working to extend R in that area.

Ajay- Companies like SAS are investing in SaaS and cloud computing. Zementis offers scored models on the cloud through PMML. Any views on just building the model or analytics on the cloud itself.

David- To me, cloud computing is a cost-effective way of dynamically scaling hardware to the problem at hand. Not everyone has access to a 20-machine cluster for high-performing computing — and even those that do can’t instantly convert it to a cluster of 100 or 1000 machines to satisfy a sudden spike in demand. REvolution R Enterprise with ParallelR is unique in that it provides a platform for creating sophisticated data analysis applications distributed in the cloud, quickly and easily.

Using clouds for building models is a no-brainer for parallel-computing problems: I recently wrote about how parallel backtesting for financial trading can easily be deployed on Amazon EC2, for example. PMML is a great way of deploying static models, but one of the big advantages of cloud computing is that it makes it possible to update your model much more frequently, to keep your predictions in tune with the latest source data.

Ajay- What are the major alliances that REvolution has in the industry.

David- We have a number of industry partners. Microsoft and Intel, in particular, provide financial and technical support allowing us to really strengthen and optimize R on Windows, a platform that has been somewhat underserved by the open-source community. With Sybase, we’ve been working on combing REvolution R and Sybase Rap to produce some exciting advances in financial risk analytics. Similarly, we’ve been doing work with Vhayu’s Velocity database to provide high-performance data extraction. On the life sciences front, Pfizer is not only a valued client but in many ways a partner who has helped us “road-test” commercial grade R deployment with great success.

Ajay- What are the major R packages that REvolution supports and optimizes and how exactly do they work/help?

David- REvolution R works with all the R packages: in fact, we provide a mirror of CRAN so our subscribers have access to the truly amazing breadth and depth of analytic and graphical methods available in third-party R packages. Those packages that perform intensive mathematical calculations automatically benefit from the optimized math libraries that we incorporate in REvolution R Enterprise. In the future, we plan to work with authors of some key packages provide further improvements — in particular, to make packages work with ParallelR to reduce computation times in multiprocessor or cloud computing environments.

Ajay- Are you planning to lay off people during the recession. does REvolution Computing offer internships to college graduates. What do people at REvolution Computing do to have fun?

David- On the contrary, we’ve been hiring recently. We don’t have an intern program in place just yet, though. For me, it’s been a really fun place to work. Working for an open-source company has a different vibe than the commercial software companies I’ve worked for before. The most fun for me has been meeting with R users around the country and sharing stories about how R is really making a difference in so many different venues — over a few beers of course!

David Smith
Director of Community
David has a long history with the statistical community.  After graduating with a degree in Statistics from the University of Adelaide, South Australia, David spent four years researching statistical methodology at Lancaster University (United Kingdom), where he also developed a number of packages for the S-PLUS statistical modeling environment. David continued his association with S-PLUS at Insightful (now TIBCO Spotfire) where for more than eight years he oversaw the product management of S-PLUS and other statistical and data mining products. David is the co-author (with Bill Venables) of the tutorial manual, An Introduction to R , and one of the originating developers of ESS: Emacs Speaks Statistics. Prior to joining REvolution, David was Vice President, Product Management at Zynchros, Inc.

Ajay – To know more about David Smith and REvolution Computing do visit http://www.revolution-computing.com and

http://www.blog.revolution-computing.com
Also see interview with Richard Schultz ,­CEO REvolution Computing here.

http://www.decisionstats.com/2009/01/31/interviewrichard-schultz-ceo-revolution-computing/

Ajay – What has been the story of your career and what message would you like to send to young people aspiring for MBA’s or just starting their careers?

Kaustubh- My first job was as a peon with SPCA, handling paperwork, dishes, etc. My Father wanted to see me getting a bicycle from my own money and that is why it happened. Thanks to Papa, I learnt some important lessons while serving people. During graduation I was doing odd jobs like a faculty at a computer institute, freelance programmer, etc.

The first experience of a large organization came @ Bharti Telecom, where I did my summers. It was a market research project and I remember sleeping in an interviewee’s cabin during a survey. After my PGDM from IIML, I got into Tech Pacific, and then ICICI then ABN AMRO. Please visit my linkedin profile for more details

My message to people doing their MBA is simple – MBA is not the end, it is just a via media for you to get into a good career. Get into an MBA because YOU want to do it and not because everyone else is doing it. There are so many careers options in front of you, follow your heart.

For people starting their careers, just 7 words – realize the power within & follow your dreams.

Ajay- Why did you create a startup? Why did you name it White Collar company ( there was an ad of a business school reunion which had the same name). What is your vision for White Collar Company
Kaustubh - When I was doing my job, I was always over achieving targets, but after some time a rut sets in. I also realized that complete freedom and maximum returns for my efforts were absent. There were so many things, ideas, etc simmering inside me but I could not do anything inside. To do all that, I had to venture on my own and venture I did. So the biggest reason I started my own company was to put my ideas into practice.

White Collar is a name generally associated with knowledge. I first wanted to name it ‘white’ but the name, domain name, trademarks etc were not available. White denotes knowledge. Our goddess of knowledge and learning ‘Saraswati’ is dressed in white. As all my ventures are essentially about knowledge and learning, so white collar. And White Collar Biker sounds cool and very oxymoronish.

I see White Collar Company to be known as the cradle of new ideas, innovation and creativity in the field of knowledge. A university is next in some years.

Ajay- What are the key learnings that you have learnt in this short period? name some companies in the United States that are similar to your company. What do you think is the market potential of this segment.
Kaustubh-We are 3 industries – adventure tourism, corporate training and hr advisory. While in the first and the last there are people doing nearly the same thing (I would not say exactly, because we do have our USPs) in corporate training – White Collar Company is the only company in the world conducting management training through motorcycles

With innovation and RoI being extremely important in training, the market potential is huge. In adventure tourism also the potential is great as we are waking upto it. In consultancy as we operate in SME space, the potential again is very large.

It has been a short period to have big learning, but I have been applying learning I had in my previous jobs to this like vendor management, marketing channel management, etc. But yes, I learnt the art of hard bargain and negotiations during this short period.

Ajay- Is an MBA (IIM or Otherwise) necessary for success. Comments please.
Kaustubh-Ajay, your question here says success. Before answering this question, I would first differentiate between 2 successes we are talking about. Success in corporate life is different from success as a entrepreneur.

For being successful as a corporate executive, MBA to a certain extent is good. It gives you certain kind of thought processes and also a platform for future success.

However, if we talk of a successful entrepreneur, I personally do not think MBA will matter much. In fact I often talk of the ‘1st of the month’ syndrome – this is the comfort of getting a handsome amount deposited as your salary every month. When you get into that comfort zone, it becomes very hard to come out. Larger the amount, harder it gets. For a successful entrepreneur – perseverance, self belief, ability of trust and ability to take risk is very important. I doubt if any MBA is going to give you that. The very same thought processes, way of thinking that help you succeed in corporate life, need to be challenged as an entrepreneur.

Ajay- Whats your vision for your web site. Which website is a good analogy for it? Why should anyone visit the website?

Kaustubh- I am not a technical person, but having said that, I see my website to be the focal point of my business. I myself built my website using widgets, etc and going forward all my business will happen from the site. By 2010, we will put a strong CRM and PRM on the website, thus enabling all business processes to be routed through the website. Like I said, I am not a techie, but I think Web 2.0, participative nature of the internet and cloud computing are going to help me save and optimize. We already have an online chat built in site, any customer can come and get more details about our programs.

Going forward, customers will be able to do bookings themselves on the site. Vendors will be able to log in do all necessary business through website and we plan to implement SFA for our employees. I believe this answers the vision and why should anyone visit my site.

Ajay-What is your favorite incident in this short period of your startup. What were the key learnings. Are you seeking venture capital funds.

Kaustubh- For customers, I thought the typical profile that will come will be young males, I was delighted when a female became our first customer. We have tweaked our marketing strategy and positioning after that.

At this stage my baby is too young and fragile. If I give her crutches to walk, she will never be able to stand up herself and be counted. So while we will go for external funding at some point of time, that time is not now. With our kind of business model, right now we are not ready for the interference of a venture capitalist.

So if you always wanted to travel to India and have an adventure as well contact Kaustubh at http://www.wccindia.com/rider/R_kaustubh.html and he will show you to be a White Rider too.

Read more about his company here – http://www.wccindia.com/rider/whywhite.html

Ajay – Describe your career in applied science. What made you decide to pursue a career in science? Some people think that careers in science are boring. How would you convince a high school student to choose a career in science?

Dominic- It’s important to realize that we are surrounded by products of science and engineering. By products of science, I mean bridges we cross on our way to work, video games we play for entertainment, or even the fabric of clothes we wear. Anyone who is curious should want to know how things really work. In that case, a scientific education makes sense, because it provides the tools necessary to understand and improve our world. I would also argue that a scientific training can also be a stepping stone towards high levels of achievements in other fields. For example, to become a financial wizard, a top patent attorney, or direct large clinical trials, a scientific education serves as a strong foundation. In addition, it’s probably easier to switch from science to another field than the other way round. Who wants to learn about matrix calculus in their forties? In my case, I graduated with a Masters in Computer Science degree, and spent 10 years at Microsoft leading software development teams for the Windows server, Exchange server, and Dynamics CRM product lines. I wish that, along the way, I had found time for a PhD in data mining, but years of practical software engineering experience also has its advantages.

Ajay- What advice would you give to someone who just got laid off, and is pondering whether he should / should not start a business?

Dominic- Working for a large company used to mean trading some autonomy for more stability and access to a wide array of resources. However, in this economy, the terms of the equation have changed. Many workers who lost their jobs found that this stability had disappeared. Others found that resources have become scarcer due to shrinking budgets. With this shift in the balance, entrepreneurship starts becoming more appealing.

Creating your own business might sound daunting, but for example creating a US Washington State LLC takes about 15 minutes, costs 200 dollars, and only requires an Internet connection. Managing payroll may sound like a big headache, but again specialized companies can handle all payroll matters on your behalf for only a few dollars a month. So while this part is relatively easy, you also need two things which are more difficult to come by:

a/ an unshakable belief in what you are trying to achieve, and

b/ a willingness to handle anything that comes your way.

You need to think like a commando solider who just landed on a beach: you’ve got great skills, but you’re alone, and can’t afford to fail. Practically, you may find yourself working for weeks or months with little or no income, and friends and family thinking that you are wasting your time. So, if necessary, try finding a co-founder to boost your confidence and motivate one another. Also, unless you want to spend most of your time chasing people for money, personal savings are a must.

Ajay- So describe your company. How does data visualization work? What differentiates your company from so many data visualization companies?

Dominic- We’re trying to stir things up a bit in terms of making it easier for regular business users to benefit from data mining. For example, we enable new “BI in the cloud” scenarios by allowing users to simply point a browser to access analysis results, or by allowing applications to submit and analyze data using an XML-based API. Built-in collaboration features, and more interactive visualizations, are also definitely part of our story.

Finally, while we focus on data mining (ex: time series forecasting, association rule mining, decision trees, etc.), we also make available other things such as pivot charts or tree maps. No data mining algorithm there, but why should business users care as long as the insight is there?

To answer your question about visualization, most packages offer basic features such as the ability to pick colors, or to change labels, etc. For differences to emerge, you have to ask the right questions.

*

Access: does visualization require an application to be installed on each computer? Our visualization work directly from a web page, so there is nothing to install (and upgrades are automatic).
*

Search: can visualization results be searched, so as to enable drill-down scenarios? In the age of Google, we enable search everywhere, so that views can be constrained to what the user is looking for.
*

Collaboration: can visualization results be tagged using comments, or shared with other users while securely controlling access, etc.? Visualization is only a starting point – chances are that you will need to talk to someone before analysis is complete – so we offer plenty of collaboration features.
*

Export:how easy is it for a business user to present analysis results to management in a way that is understandable? We make it easy to export visualization content to a shared gallery, and as presentation-ready images.

There are a couple of other things we do as well in terms of interaction (ex: zoom, select, focus, smart graph layout), and a couple we don’t have yet (ex: geo-mapping, export to PDF).

But in conclusion, I would say that useful data visualization is as much about the way you present data (and that must be compelling!), as it is about how one accesses, searches, secures, shares, or exports visualizations.

Ajay- The technology sector was hit the hardest by the immigration of skilled workers. As a technology worker, what do you have to say about immigration? What do you have to say about outsourcing? Do you have any plans for selling your products outside the United States?

Dominic- I am a US permanent resident, half French, half British, and my wife is Indian. So you won’t find it surprising to hear that I am in favor of immigration. In 1996, as an engineering student in France, I made the unusual choice to study one year at the Indian Institute of Technology (Delhi).

In fact, I was the only one in my engineering college (France’s largest) to select India as a destination (my friends all went to the US, UK, Australia, Germany, etc.). Now that India has become a recognized player in the IT field, several dozen students from the same engineering college chose India as a destination. So I guess the immigration is starting to flow both ways!

Also, among the people I used to work with at Microsoft and who left to start a company, a good proportion are immigrants. So it’s important to recognize that immigrants not only help fill high-tech positions, but also create jobs.

Finally, as an entrepreneur trying to keep costs low, outsourcing is a tool you can’t afford to ignore. For example, websites such as www.elance.com provide easy access to the global marketplace. For those worried about quality, it’s possible to review customer ratings and portfolios. We keep track of visitors coming to our website, and the majority of the visitors to date have been from outside the US.

Ajay-  What is the basic science used by your company’s product?

Dominic - We use a client / server model. On the server, at the lowest level, we use SQL databases (accessed using ODBC), acting as data and configuration repositories.

Immediately above that sits a computing layer, which offers scalable, distributed data mining algorithms. We implement algorithms which scale well with the number of rows and attributes, but also properly handle a mix of discrete / numeric / missing values.

For example, just for clustering, the literature has some incredibly powerful algorithms (ex: WaveCluster, an algorithm based on wavelet transforms), but which also fail as soon as you enter real-world situations (ex: some fields are discrete).

On top of the computing layer sits a rich, secure web-based XML API, which allows users to manipulate analysis and collaboration objects, while enforcing security.

For the client, we built a web-based visualization application using Microsoft Silverlight. To ensure client / server communications are as efficient as possible, we use a fair amount of data compression and caching.

Ajay-  Who are your existing clients and what is the product launch plan for next year?

Dominic- We’re only in alpha mode right now, so our next customers are in fact beta testers. We’re still busy adding new features. It’s good to be small and nimble, it allows us to move quickly. Sorry, I can’t confirm any launch date yet!

Ajay-  What does the CEO of a startup company do, when he has free time (assuming he has any)?

Dominic- When you spend most of your time working on analytics, it’s sometimes hard to leave your analytical brain at work.

For example, I am sure that readers who come to your website and visit a casino can’t help themselves and immediately start calculating the exact odds of winning (instead of just having fun).

Among other things, I enjoy challenging friends to programming puzzles (actually, they’re recycled Microsoft interview questions). My current bedtime reading is a book about data compression. I think you got the picture!

******************************************************************

Dominic is currently making promising data visualization products at http://data-applied.com/ .To read more about him, please visit his profile page http://www.analyticbridge.com/profile/DominicPouzin

Ajay- Describe your career journey. How would you motivate your children or young people to follow careers in science or at least to pay more attention to science subjects. What advice would you give to young tech entrepreneurs in this recession- the ones chasing dreams on iMobile Applications, cloud computing etc.

Ron- Science and a curious mind go together. I remember when I first met a friend of mine who is a professor of cognitive sciences at the University of California. To me, he represents the quest for scientific knowledge. Not only has he been studying visual space perception, visual control of locomotion, and spatial cognition, but he is also interested in every single aspect of the world around him. I believe that if we are genuinely interested and curious to know how and why things are the way they are, we are a step closer into appreciating and willing to participate in the collective quest for scientific knowledge.

Our current economic troubles are not affecting a single industry. The problem is widespread. So, tech entrepreneurs should not view this recession as target towards technology. It is new technology in clean, renewable fuels which will most probably define what is to come. I am also old enough to know that everything is cyclical and so, this recession will lead us to great progress. iMobile Applications and Cloud Computing are here to stay since these are technologies that just make sense. Cloud Computing benefits from the pay-as-you-go model, which because of its affordability is bound to allow for the widespread use and availability of computing where we have not seen before.

The most interesting and satisfying effect one can have is transformation – do that which changes people’s lives, and your own at the same time.  I like the concept of doing well and doing good at the same time.  My emphasis has always marketing and sales in every business in which I have been involved.  ADAPA provides for delivering on the promise of predictive analytics – decisioning in real-time.

Ajay-  How do you think Cloud Computing will change the modeling deployment market by 2011. SAS Institute is also building a 70 million dollar facility for private clouds. Do you think private clouds with tied in applications would work.

Ron- Model deployment in the cloud is already a reality. By 2011, we project that most models will be deployed in the cloud (private or not). With time though, private clouds will most probably need to embrace the use of open standards such as PMML. I believe open standards such as PMML, which allows for true interoperability, will become widespread among the data mining community; be used in any kind of computing environment; and, be moved from cloud to cloud.

Ajay- I am curious- who is Zementis competition in cloud deployed models. Where is ADAPA deployment NOT suitable for scoring models – what break off point does size of data make people realize that cloud is better than server. Do you think Internal Organization IT Support teams fear cloud vendors would take their power away.

Ron- Zementis is the first and only company to provide a scoring engine on the cloud. Other data mining companies have announced their intention to move to cloud computing environments. The size of the data you need to score is not something that should be taken into account for determining if scoring should be done in the cloud or not. In ADAPA, models can uploaded and managed through an intuitive web console and all virtual machines can be launched or terminated with the click of a mouse. Since ADAPA instances run from $0.99/hour, it can appeal to small and large scoring jobs. For small, the cost is minimal and deployment of models is fast. For large, the cloud offers scalability. Many ADAPA instances can be set to run at the same time.

[youtube=http://www.youtube.com/watch?v=8hNqxqrdXLI&hl=en&fs=1]

Cloud computing is changing the way models are deployed, but all organizations still need to manage their data and so IT can concentrate on that. Scoring on the cloud makes the job of IT easier.

Ajay- Which is a case where ADAPA deployment is not suited. Software like from KXEN offers model export into many formats like PMML, SQL, C++ , SAS etc. Do you think Zementis would be benefited if it had such a converter like utility/collection of utilities on its site for the PMML conversion say from SAS code to PMML code etc. Do you think PMML is here to stay for a long time.

Ron- Yes, PMML is here to stay. Version 4.0 is about to be release. So, this is a very mature standard embraced by all leading data mining vendors. I believe the entire community will benefit from having converters to PMML, since it allows for models to be represented by an open and well documented standard. Also, since different tools already import and export PMML, data miners and modelers are the set free to move their models around. True interoperability!

Ajay – Name some specific customer success stories and costs saved.

Ron - As a team, we spent our early development time working on assignments in the mortgage business.  That’s what gave rise to the concept of ADAPA – enabling smart decisions as an integral part of the overall business strategy.  It became obvious to us that we were in fact totally horizontal with application in any industry that had knowledge to be gained from its data.  If only they could put their artful predictive models to work – easily integrated and deployed, able to be invoked directly from the business’ applications using web services, with returned results downloaded for further processing and visualization.  There is no expensive upfront investment in software licenses and hardware; no long-term extended implementation and time-to-production.  The savings are obvious, the ROI pyrotechnic.

Our current users, both enterprise installations and Amazon EC2 subscribers report great results, and for a variety of good reasons we tend to respect their anonymity:

Zementis ADAPA Case Study #1:

Financial Institution Embraces Real-time Decisions.

Decision Management:  A leading financial company wanted to implement an enterprise-wide decision system to automate credit decisions across Retail, Wholesale, and Correspondent business channels. A key requirement for the company’s Enterprise strategy was to select a solution which could execute and manage rules as well as predictive analytic
s on demand and in real-time. With minimal prior automation in place, the challenge was to execute guidelines and pricing for a variety of business scenarios. Complex underwriting and intricate pricing matrices combined present obstacles for employees and customers in correctly assessing available choices from a myriad of financial products. Although embracing a new processing paradigm, the goal for integration of the solution with the existing infrastructure also was to ensure minimal impact to already established processes and to not jeopardize origination volume.

Following a comprehensive market review, the financial institution selected the Zementis ADAPA Enterprise Edition because of its key benefits as a highly scalable decision engine based on open standards. The ADAPA framework, they concluded, ensures real-time execution capabilities for rules and predictive analytics across all products and all business channels.

Working directly with senior business and IT management, Zementis efficiently executed on an iterative deployment strategy which enabled the joint project team to roll out a comprehensive Retail solution in less than three months. Accessed in Retail offices across the country, the ADAPA decision engine assists more than 700 loan officers to determine eligibility of a borrower with the system instantly displaying conditions or exceptions to guidelines as well as precise pricing for each scenario. The Wholesale division exposes the ADAPA decision engine to a large network of several thousand independent brokers who explore scenarios and submit their applications online. While rules were authored in Excel format, a favorite of many business users, predictive models were developed in various analytics tools and deployed in ADAPA via the Predictive Model Markup Language (PMML) standard. Extending its value across the entire enterprise, ADAPA emerged as the central decision hub for vital credit, risk, pricing, and other operational decisions.

Zementis ADAPA Case Study #2:

Delivering Predictive Analytics in the Cloud.

A specialized consulting firm with a focus on predictive analytics needed a cost-effective, agile deployment framework to deliver predictive models to their clients.  The firm specializes in outsourcing the development of predictive models for their clients, using various tools like R, SAS, and SPSS. Supporting open standards, the natural choice was to utilize the Predictive Model Markup Language (PMML) to transfer the models from the scientist’s development environment to a deployment infrastructure.  One key benefit of PMML is to remain development tool agnostic.  The firm selected the Zementis ADAPA Predictive Analytics Edition on the Amazon Elastic Compute Cloud (Amazon EC2) which provides a scalable, reliable deployment platform based on the PMML standard and Service Oriented Architecture (SOA).

With ADAPA, the firm was able to shorten the time-to-market for new models delivered to clients from months to just a few hours.  In addition, ADAPA enables their clients to benefit from a cost-effective SaaS utility-model, whereby the Zementis ADAPA engine is available on-demand at a fraction of the cost of traditional software licenses, eliminating upfront capital expenditures in both hardware and software. The ADAPA Predictive Analytics Edition has given the firm a highly competitive model delivery process and its clients an unprecedented agility in the deployment and integration of predictive analytics in their business processes.

Zementis ADAPA Case Study #3:

Assessing Risk in Real-Time for On-Line Merchant.

An on-line merchant with millions of customers needed to assess risk for submitted transactions before being sent to a credit-card processor.  Following a comprehensive data analysis phase, several models addressing specific data segments were built in a well-know model development platform.  Once model development is complete, models are exported in the PMML (Predictive Model Markup Language) standard. The deployment solution is the ADAPA Enterprise Edition, using its capabilities for data segmentation, data transformation, and model execution. ADAPA was selected as the optimal choice for deployment, not only because PMML-based models can easily be uploaded and are available for execution in seconds, but also because ADAPA Enterprise edition offers the seamless integration of rules and predictive analytics within a single Enterprise Decision Management solution.

ADAPA was deployed on-site and configured to handle high-volume, mission-critical transactions.  The firm not only leveraged the real-time capabilities of ADAPA, but also its integrated reporting framework.  It was very important for the merchant to assess model impact on credit card transactions on a daily basis. Given that ADAPA allows for reports to be uploaded and managed via its web administration console, the reporting team was able to design new reports, schedule them for routine execution, and send the results in PDF format for analysis to the business department with the required agility. During the implementation of the roll-out strategy, the ADAPA web console and its ease of use allowed for effective management of rules and models as well as active monitoring of deployed models and impact of decisions on the business operation.

 

For More on Zementis see here www.zementis.com

Ajay: Describe your career in Science. Are careers in science less lucrative than careers in business development? What advice would you give to people re-skilling in the current recession on learning analytical skills?

Olivier: I have a Master of Science in Geophysics and Master of Science in Computer Sciences, both from Paris VI University. I have always tried to combine science and business development in my career as I like to experience all aspects – from idea to concept to business plan to funding to development to marketing to sales.

There was a study published earlier this year that said two of the three best jobs are related to math and statistics. This is reinforced by three societal forces that are converging – better uses of mathematics to drive decision making, the tremendous growth and storage of data, and especially in this economy, the ability to deliver ROI. With more and more commercial and government organizations realizing the value of Predictive Analytics to solve business problems, being equipped with analytical skills can only enhance your career and provide job security.

Ajay: So SPSS has launched new products within its Predictive Analytics Software (PASW) portfolio – Modeler 13 and Text Analytics 13? Is this old wine in a new bottle? What is new in terms of technical terms? What is new in terms of customers looking to mine textual information?

Olivier: Our two new products — PASW Modeler 13 (formerly Clementine) and PASW Text Analytics 13 (formerly Text Mining for Clementine) – extend and automate the power of data mining and text analytics to the business user, while significantly enhancing the productivity, flexibility and performance of the expert analyst.

PASW Modeler 13 data mining workbench has new and enhanced functionality that quickly takes users through the entire data mining process – from data access and preparation to model deployment. Some the newest features include Automated Data Preparation that conditions data in a single step by automatically detecting and correcting quality errors; Auto Cluster that gives users a simple way to determine the best cluster algorithm for a particular data set; and full integration with PASW Statistics (formerly SPSS Statistics).

With PASW Text Analytics 13, SPSS provides the most complete view of the customer through the combined analysis of text, web and survey data.   While other companies only provide the text component, SPSS couples text with existing structured data, permitting more accurate results and better predictive modeling. The new version includes pre-built categories for satisfaction surveys, advanced natural language processing techniques, and it supports more than 30 different languages.

Ajay: SPSS has supported open source platforms – Python and R – before it became fashionable to do so. How has this helped your company?

Olivier: Open source software helps the democratization of the analytics movement and SPSS is keen on supporting that democratization while welcoming open source users (and their creativity) into the analytics framework.

Ajay: What are the differences and similarities between Text Analytics and Search Engines? Can we mix the two as well using APIs?

Olivier: Search Engines are fundamentally top-down in that you know what you are looking for when launching a query. However, Text Analytics is bottom-up, uncovering hidden patterns, relationships and trends locked in unstructured data – including call center notes, open-ended survey responses, blogs and social networks. Now businesses have a way of pulling key concepts and extracting customer sentiments, such as emotional responses, preferences and opinions, and grouping them into categories.

For instance, a call center manager will have a hard time extracting why customers are unhappy and churn by using a search engine for millions of call center notes. What would be the query? But, by using Text Analytics, that same call center agent will discover the main reasons why customers are unhappy, and be able to predict if they are going to churn.

Ajay: Why is Text Analytics so important?  How will companies use it now and into the future?
Olivier - Actually, the question you should ask is, "Why is unstructured data so important?" Today, more than ever, people love to share their opinions — through the estimated 183 billion emails sent, the 1.6 million blog posts, millions of inquiries captured in call center notes, and thousands of comments on diverse social networking sites and community message boards. And, let’s not forget all data that flows through Twitter. Companies today would be short-sighted to ignore what their customers are saying about their products and services, in their own words. Those opinions – likes and dislikes – are essential nuggets and bear much more insights than demographic or transactional data to reducing customer churn, improving satisfaction, fighting crime, detecting fraud and increasing marketing campaign results.

Ajay: How is SPSS venturing into cloud computing and SaaS?

Olivier: SPSS has been at the origin of the PMML standard to allow organizations to provision their computing power in a very flexible manner – just like provisioning computing power through cloud computing. SPSS strongly believes in the benefits of a cloud computing environment, which is why all of our applications are designed with Service Oriented Architecture components.  This enables SPSS to be flexible enough to meet the demands of the market as they change with respect to delivery mode.  We are currently analyzing business and technical issues related to SPSS technologies in the cloud, such as the scoring and delivery of analytics.  In regards to SaaS, we currently offer hosted services for our PASW Data Collection (formerly Dimensions) survey research suite of products.

Ajay: Do you think business intelligence is an over used term? Why do you think BI and Predictive Analytics failed in mortgage delinquency forecasting and reporting despite the financial sector being a big spender on BI tools?

Oliver: There is a big difference between business intelligence (BI) and Predictive Analytics. Traditional BI technologies focus on what’s happening now or what’s happened in the past by primarily using financial or product data. For organizations to take the most effective action, they need to know and plan for what may happen in the future by using people data – and that’s harnessed through Predictive Analytics.

Another way to look at it – Predictive covers the entire capture, predict and act continuum – from the use of survey research software to capture customer feedback (attitudinal data), to creating models to predict customer behaviors, and then acting on the results to improve business processes. Predictive Analytics, unlike BI, provides the secret ingredient and answers the question, “What will the customer do next?”

That being said, financial institutions didn’t need to use Predictive Analytics to
see
that some lenders sold mortgages to unqualified individuals likely to default. Predictive Analytics is an incredible application used to detect fraud, waste and abuse. Companies in the financial services industry can focus on mitigating their overall risk by creating better predictive models that not only encompass richer data sets, but also better rules-based automation.

Ajay: What do people do at SPSS to have fun when they are not making complex mathematical algorithms?
Oliver: SPSS employees love our casual, friendly atmosphere, our professional and talented colleagues, and our cool, cutting-edge technology. The fun part comes from doing meaningful work with great people, across different groups and geographies. Of course being French, I have ensured that my colleagues are fully educated on the best wine and cuisine. And being based in Chicago, there is always a spirited baseball debate between the Cubs and White Sox. However, I am yet to convince anyone that rugby is a better sport.

Biography

Olivier Jouve is Vice President, Corporate Development, at SPSS Inc. He is responsible for defining SPSS strategic directions, growth opportunities through internal development, merger and acquisitions and/or tactical alliances. As a pioneer in the field of data and text mining for the last 20 years, he has created the foundation of Text Analytics technology for analyzing customer interactions at SPSS. Jouve is a successful serial entrepreneur and has had his works published internationally in the area of Analytical CRM, text mining, search engines, competitive intelligence and knowledge management.

From the Knime.com website

The modular data exploration platform KNIME, originally solely developed at the University of Konstanz, Germany, enables the user to visually create data flows – or pipelines, execute selected analysis steps, and later investigate the results through interactive views on data and models. KNIME already has more than 2,000 active users in diverse application areas, ranging from early drug discovery and customer relationship analysis to financial information integration.

Ajay – What prompted you personally to be part of KNIME and not join a big technology  company?  What does the future hold for KNIME in 2009-10?

Fabian -I was excited when I first joined the KNIME team in 2005. Back then, we were working exclusively on the open source version backed by some academic funding. Being part of the team that put together such a professional data mining environment from scratch was a great experience. Growing this into a commercial support and development arm has been a thrill as well. The team and the diverse experiences gained from helping get a new company off the ground and being involved in everything it takes to enable this to be successful made it unthinkable for me to work anywhere else.

We continue to develop the open source arm of KNIME and many new features lie ahead: text, image, and time series processing as well as better support for variables. We are constantly working on adding new nodes. KNIME 2.1 is expected in the fall and some of the ongoing development can already be found on the KNIME Labs page (http://labs.knime.org)

The commercial division is providing support and maintenance subscriptions for the freely available desktop version. At the same time we are developing products which will streamline the integration of KNIME into existing IT infrastructures:

• the KNIME Grid Support lets you run your compute-intensive (sub-) workflows or nodes on a grid or cluster;

• KNIME Reporting makes use of KNIME’s flexibility in order to gather the data for your report and provides simplified views (static or interactive=dashboards) on the resulting workflow and its results; and

• the KNIME Enterprise Server facilitates company-wide installation of KNIME and supports collaboration between departments and sites by providing central workflow repositories, scheduled and remote execution, and user rights management.

Ajay -Software as a service and Cloud Computing is the next big thing in 2009. Are there any plans to put KNIME on a cloud computer and charge clients for the hour so they can build models on huge data without buying any hardware but just rent the time?

Fabian – Cloud computing is an agile and client-centric approach and therefore fits nicely into the KNIME framework, especially considering that we are already working on support for distributed computing of KNIME workflows (see above). However, we have no immediate plans for KNIME workflow processing on a per-use charge or similar. That’s an interesting idea, though. The way KNIME nodes are nicely encapsulated (and often even distributable themselves) would make this quite natural.

Ajay – What differentiates KNIME from other products such as RPro and Rapid Miner, for example? What are the principal challenges you have faced in developing it? Why do customers like and dislike it?

Fabian- Every tool has its strengths and weaknesses depending on the task you actually want to accomplish. The focus of KNIME is to support the user during his or her quest of understanding large and heterogeneous data and to make sense out of it. For this task, you cannot rely only on classical data mining techniques, wrapping them into a command line or otherwise configurable environment, but simple, intuitive access to those tools is required in addition to supporting visual exploration with interactive linking and brushing techniques.

By design, KNIME is a modular integration platform, which makes it easy to write own nodes (with the easy-to-use API) or integrate existing libraries or tools.

We integrated Weka, for example, because of its vast library of state-of-the-art machine learning algorithms, the open source program R – in order to provide access to a rich library of statistical functions (and of course many more) – and parts of the Chemistry Development Kit (CDK). All these integrations follow the KNIME requirements for easy and intuitive usage so the user does not need to understand the details of each tool in great depth.

A number of our commercial partners such as Schroedinger, Infocom, Symyx, Tripos, among others, also follow this paradigm and similarly integrate their tools into KNIME. Academic collaborations with ETH Zurich, Switzerland on the High Content Screening Platform HC/DC represent another positive outcome of this open architecture. We believe that this strictly result-oriented approach based on a carefully designed and professionally coded framework is a key factor of KNIME’s broad acceptance. I guess this is another big differentiator: right from the start, KNIME has been developed by a team consisting of SW developers with decades of industrial SW engineering experience.

Ajay – Any there any Asian plans for KNIME? Any other open source partnerships in the pipeline?

Fabian – We have a Japan-based partner, Infocom, who operates in the fields of life science. But we are always open for other partnerships, supporters, or collaborations.

In addition to the open source integrations mentioned above (Weka, R, CDK, HC/DC), there are many other different projects in the works and partnerships under negotiation. Keep an eye on our blog and on our Labs@KNIME page (labs.knime.org).

ABOUT

KNIME – development started in January 2004. Since then: 10 releases; approx. 350,000 lines of code; 25,000 downloads; an estimated 2000 active users. KNIME.com was founded in June 2008 in Zurich, Switzerland.

Fabian Dill – has been working for and with KNIME since 2005; co-founder of KNIME.com.

Here is an interview with the head of marketing of Visual Numerics, Alicia McGreevey.

Visual Numerics® is the leading provider of data analysis software, visualization solutions and expert consulting for technical, business and scientific communities worldwide (see www.vni.com ).

Ajay – Describe your career in science so far. How would explain embeddable analytics to a high school student who has to decide between getting a MBA or a Science degree.

Alicia - I think of analytics as analyzing a situation so you can make a decision. To do that objectively, you need data about your situation. Data can be anything: foreign currency exchange rates, the daily temperature here in Houston, or Tiger Wood’s record at the Master’s tournament when he’s not leading after the 3^rd round.

Embedding analytics is simply making the analysis part of an application close to, or embedded with, your data. As an example, we have a customer in Germany, GFTA (Gesellschaft Fuer Trendanalysen), who has built an application that embeds analytics to analyze historic and live tick foreign exchange rate data. Their application gives treasuries and traders predictions on what is about to happen to exchange rates so they can make good decisions on when to buy or sell.

Embedding analytics is as much a business discipline as it is science. Historically, our analytics have been used predominantly by the government and scientific community to perform heavy science and engineering research. As business intelligence becomes increasingly important to compete in today’s marketplace, our analytics can now be found driving business decisions in industries like financial services, healthcare and manufacturing. Partners like Teradata and SAP are embedding our analytics into their software as a way to extend their current offerings. As their customers demand more custom BI solutions to fit unique data sets, our analytics provide a more affordable approach to meet that need. Customers now have an option to implement custom BI without incurring the massive overhead that you would typically find in a one-size-fits-all solution.

If you’re a student, I’d recommend you invest time and course work in the area of analytics regardless of the discipline you choose to study. The term analytics is really just a fancy term for math and statistics. I’ve taken math and statistics courses as part of a science curriculum and as part of a business curriculum. Being able to make optimal decisions by objectively analyzing data is a skill that will help you in business, science, engineering, or any area.

Ajay – You have been working behind the scenes quietly building math libraries that power many partners. Could you name a few success stories so far.

Alicia - One of the most interesting things about working at Visual Numerics is our customers. They create fascinating analytic applications using mathematic and statistical functions from our libraries. A few examples:

• Total, who you probably know as one of the world’s super major oil companies, uses our math optimization routines in an application that automatically controls the blending of components in the production of gasoline, diesel and heavy fuels. By making best use of components, Total helps minimize their refining costs while maximizing revenue.

• The Physics Department at the University of Kansas uses nonlinear equation solvers from our libraries to develop more efficient particle beam simulations. By simulating the behavior of particle beams in particle accelerators, scientists can better design particle accelerators, like the LHC or Large Hadron Collider, for high-energy research.

• A final example that I think is interesting, given the current economic situation, is from one of our financial customers RiskMetrics Group. RiskMetrics uses functions from our libraries to do financial stress testing that allows portfolio fund managers simulate economic events, like the price of oil spiking 10% or markets diving 20%. They use this information to predict impacts on their portfolio and make better decisions for their clients.

Ajay – What have been the key moments in Visual Numerics path so far.

Alicia – Our company has been in business for over 38 years, rooted in the fundamentals of mathematics and statistics. It started off as IMSL, offering IMSL Numerical Libraries as a high performance computing tool for numerical analysis. Before visualization was fashionable, we saw visualization as an important part of the data analysis process. As a result, the company merged with Precision Visuals, makers of PV-WAVE (our visual data analysis product) in the 1990s to become what is now known as Visual Numerics.

Looking back at recent history, a major event for Visual Numerics was definitely when SAP AG licensed the libraries at the end of 2007. For several years leading up to 2007, we’d seen increased interest in our libraries from independent software vendors (ISVs). More and more ISVs with broad product offerings were looking to provide their customers with analytic capabilities, so we had invested considerably in making the libraries more attractive to this type of customer. Having SAP, one of the largest and most respected ISVs in the world, license our products gave us confidence that we could be a valued OEM partner to this type of customer.

Ajay – What are the key problems you face in your day to day job as a Visual Numerics employee. How do you have fun when not building math libraries.

Alicia – In marketing, our job is to help potential users of our libraries understand what it is we offer so that they can determine if what we offer is of value to them. Often the hardest challenge we face is simply finding that person. Since our libraries are embeddable, they’ve historically been used by programmers. So we’ve spent a lot of time at developer conferences and sponsoring developer websites, journals and academic programs.

One product update this year is that we’ve made the libraries available from Python, a dynamic scripting language. Making IMSL Library functions available from Python basically means that someone who is not a trained programmer can now use the math and stats capabilities in the IMSL Libraries just like a C, Java, .Net or Fortran developer. It’s an exciting development, though brings with it the challenge of letting a whole new set of potential users know about the capabilities of the libraries. It’s a fun challenge though.

On a more fun side of things, you may be interested to know that our expertise in math and statistics led us to some Hollywood fame. At one point in time, we were selected to review scripts for the crime busting drama, NUMB3RS. NUMB3RS, aired on CBS in the US and features an FBI Special Agent who recruits his brilliant mathematician brother to use the science of mathematics with its complex equations to solve the trickiest crimes in Los Angeles. So yes, the math behind the Show is real and it is exciting indeed to see how math can be applied in all aspects of our lives, including ferreting out criminals on TV!

Ajay -What is the story ahead. How do you think Visual Numerics can help demand forecasting and BI to say BYE to the recession.

We’re seeing more success stories from customers using analytics and data to make good decisions and I think the more organizations leverage analytics, the faster we’ll emerge from this economic slump.

As an example, we have a partner, nCode International, who makes software to help manufacturers collect and analyze test data and use the analysis to make design decisions. Using it, automobile manufacturers can, for example, analyze real-world driving pattern data for different geographic areas (e.g., emerging markets like China and India versus established markets like the USA and Europe) and design the perfect vehicle for specific markets.

So the analytic successes are out there and we know that organizations have multitudes of data. Certainly every organization that we work with has more data today than ever before. For analytics to help us say Bye to the recession, I think we need to continue to promote our successes, make analytic tools available to more users, and get users across multiple disciplines and industries using analytics to make the best possible decisions for their organizations.

Personal Biography:

As Director of Marketing for Visual Numerics, Alicia is an authority on how organizations are using advanced analytics to improve performance. Alicia brings over 15 years of experience working with scientists and customers in the planning and development of new technology products and developing go to market plans. She has a B.A. in Mathematics from Skidmore College and an M.B.A. from the University of Chicago Booth School of Business.

]]> http://www.dudeofdata.com/decisionstats/?feed=rss2&p=1733 1 http://www.dudeofdata.com/decisionstats/?p=1450 http://www.dudeofdata.com/decisionstats/?p=1450#comments Fri, 27 Mar 2009 17:44:01 +0000 ajayohri http://www.decisionstats.com/2009/03/interview-franoise-soulie-fogelman-kxen/ This week KXEN launched it’s social network analysis tool thus gaining a unique edge in being the first to launch social network tools for analytics. Having worked with KXEN as an analyst for scoring model- I am aware of the remarkable innovations they bring to their premium products. In an exclusive interview ,KXEN’s Vice President for  Strategic Business Development, Françoise Soulie Fogelman agreed to share some light on this remarkable new development in statistical software development.

Ajay – Françoise, how does the Social Network Analysis module helps model building for marketing professionals.

Françoise- KXEN Social network Analysis module (KSN) helps build models which take into account interactions between customers. This is done in 3 stages :

• The data describing interactions is used to build a social network structure (actually, usually various social network structures are built in one pass through the data). You can explore your network to understand better the behavior of a given customer and what is happening around him.
• From each social network structure, a set of attributes is automatically built by KSN for each node: it could be number of neighbors, average value of a given customer attribute among neighbors … Actually, you can have statistics on anything you have loaded into the system as customer node decoration. Usually, you’ll generate at this stage a few tens of social attributes per social network structure.
• You then join these social attributes to the existing customer attributes. After that, you build your model as usual.

Ajay – But how does the KSN module work and which mathematical technique is it based on (or is it just addition of extra variables). Are there any proprietary patents that KXEN have filed in this field (both automated modeling as well as social network analysis).

Françoise- The KSN module uses (for extracting social attributes) graph theory. KXEN has not filed a patent in relation with KSN.

Ajay – There are many modeling software but very few which involve social network analysis though many companies have expressed interest in this. What are the present rivals to KSN module specifically in software and who do you think the future rivals will be?

Françoise- There are many software tools, but when it comes to the ability to handle very large graphs, not very many are left. We consider that our only real competitor today is SAS who has an offer for Social network Analysis, but this product is specifically targeted for fraud in bank and insurance. There are also companies positioned in Telco, usually offering a consulting service, built around an internal product. We think our solution is unique in its ability to handle very large volumes (we’re talking here more than 40 M nodes and 300 M links) and to address all industry domains. As usual, we offer a tool which is an exploratory tool, giving the customer the ability to produce by himself as many models as he wants.

Ajay – Who would be the typical customer or potential clients for KSN module? In which domains would this module be not so relevant? Are there any specific case studies that you can point out?

Françoise- This is a first version, so we do not really know yet who the typical customer will be and cannot point yet to case studies. However, Telco operators have expressed a very strong interest and we already have a Telco customer with whom we’ve worked on marketing projects. So our first case studies will most certainly come from Telco. We are working on some research projects in the retail space. We think that banks (for fraud), social sites, blogs sites and forums will be our next customers. The sector where I do not see (yet?) a potential is manufacturing industries.

Ajay – How would privacy concerns of customers be addressed with the kind of social network analysis that KSN can now offer to marketers.

Françoise- KXEN offers a tool to build models and is not concerned with the problem of collecting, storing and exploiting data: this is KXEN customer’s responsibility. Depending upon the country, there are various jurisdictions protecting the storage and use of data and those will naturally apply to building and analyzing Social Networks. However, in the case of Social Network Analysis the issue of “ethical” use will be more sensitive.

Ajay –What kind of hardware solutions go best with KXEN’s software. What are the other BI vendors that your offerings best complement with.

Françoise - KXEN software in general and KSN in particular, run on any platform. When using KSN to build decent size graphs (with tens of millions of nodes and hundreds of millions of links for example), 64 bits architecture is required. A recent survey of KXEN customers show that the BI suites used by our customers are mostly MicroStrategy and Business Objects (SAP). We also like very much to mention Advizor Solutions which offers data visualization software already embedding KXEN technology.

Ajay –Do you think the text mining as well as the Data Fusion approach can work for online web analytics, search engines or ad targeting?

Françoise -Of course, our data fusion approach can be very well suited for online web analytics and ad targeting (we have a number of partners that either are already using KXEN for this purpose or developing applications in these domains using KXEN technology). We would be more cautious for search engines per se.

Ajay –Are there any plans for offering KXEN products as a Service (like Salesforce.com) instead of the server based approach?

Françoise - We do not have yet plans to offer KXEN products as a service yet, but, again, we have partners such as Kognitio that offers analytics platforms embedding KXEN.

Brief Biography-

Françoise Soulie Fogelman is responsible for leading KXEN business development, identifying new business opportunities for KXEN and working with Product development, Sales and Marketing to help promote KXEN’s offer. She is also in charge of managing KXEN’s University Program.

Ms Soulie Fogelman has over 30 years of experience in data mining and CRM both from an academic and a business perspective. Prior to KXEN, she directed the first French research team on Neural Networks at Paris 11 University where she was a CS Professor. She then co-founded Mimetics, a start-up that processes and sells development environment, optical character recognition (OCR) products and services using neural network technology, and became its Chief Scientific Officer. After that she started the Data Mining and CRM group at Atos Origin and, most recently, she created and managed the CRM Agency for Business & Decision, a French IS company specialized in Business Intelligence and CRM.

Ms Soulie Fogelman holds a master’s degree in mathematics from Ecole Normale Superieure and a PhD in Computer Science from University of Grenoble. She was advisor to over 20 PhD on data mining, has authored more than 100 scientific papers and books and has been an invited speaker to many academic and business events.


0;

   ( Ajay – So it seems like an interesting software and with the marketing avenues for social networking growing, and analytics modelers exploring the last bit of data for incremental field – this is an area where we can be sure of new developments soon. I wonder what the response from other analytics vendors including open source developers would be as this does seem a promising area for statistical modelling as well as analysis. What do you think ?? Can I search all data from Twitter , Facebook ,search results on Indeed .com and Linkedin and add it to your credit profile for creating a better propensity model .. Will the credit or marketing behavior scores of your friends affect your propensity and thus the telecom ads you see while surfing … …)

I was in the middle of interviewing people as well as helping the good people in my new role as a community evangelist at Smart Data Collective when I got a LinkedIn Request to join the SDC group  from Jon Peck .

SPSS Inc. is a leading worldwide provider of predictive analytics software and solutions. Founded in 1968, today SPSS has more than 250,000 customers worldwide, served by more than 1,200 employees in 60 countries .Now Jon is a legendary SPSS figure and a great teacher in this field .I asked him for an interview he readily agreed.

Jon Peck is a Principal Software Engineer and Technical Advisor at SPSS. He has been working with SPSS since 1983  and in the interview he talks from the breadth of his perspective and experience on things in analytics and at SPSS .

Ajay - Describe your career journey from college to today. What advice would you give to young students seeking to be hedge fund managers rather than scientists.  What are the basic things that a science education can help students with , in your opinion ?

Jon- After graduating from college with a B.A. in math, I earned a Ph. D in Economics, specializing in econometrics, and taught at a top American university for 13 years in the Economics and Statistics Departments and the School of Organization and Management.  Working in an academic environment all that time was a great opportunity to grow intellectually.  I was increasingly drawn to computing and eventually decided to join a statistical software company.  There were only two substantial ones at the time.  After a lot of thought, I joined SPSS as it seemed to be the more interesting place and one where I would be able to work in a wider variety of areas.  That was over 25 years ago!  Now I have some opportunities to teach and speak again as well as working in development, which I enjoy a lot.

I still believe in getting a broad liberal arts education along with as much quantitative training as possible.  Being able to work in very different areas has been a big asset for me.  Most people will have multiple careers, so preparing broadly is the most important career thing you can do.  As for hedge fund jobs – if there are any left, I’d say not to be starry-eyed about the money.  If you don’t choose a career that really interests you, you won’t be very successful anyway. Do what you love – subject to earning a living.

Math and scientific reasoning skills are preparation for working in many areas as well as being helpful in making the many decisions with quantitative aspects in life.  Math, especially, provides a foundation useful in many areas.  The recently announced program in the UK to improve general understanding of probability illustrates some practical value.

Ajay- What are SPSS’s contribution to Open Source software . What ,if you can disclose are any plans for further increasing that involvement.

Jon-  I wish I could talk about SPSS future plans, but I can’t.  However, the company is committed to continuing its efforts in Python and R.  By opening up the SPSS technology with these open source technologies, we are able to expand what we and our users can do.  At the same time, we can make R more attractive through nicer output and simpler syntax and taking away much of the pain.  One of the things I love about this approach is how quickly and easily new things can be produced and distributed this way compared to the traditional development cycle.  I wrote about productivity and Python recently on my blog at insideout.spss.com.

Ajay – How happy is the SPSS developer community with Python . Are there any other languages that you are considering in the future.

Jon- Many in the SPSS user community were more used to packaged procedures than to programming (except in the area of data transformations).  So Python, first, and then R were a shock.  But the benefits are so large that we have had an excellent response to both the Python and R technologies.  Some have mastered the technology and have been very successful and have made contributions back to the SPSS community.  Others are consumers of this technology, especially through our custom dialogs and extension commands that eliminate the need to learn Python or R in order to use programs in these languages.  Python is an outstanding language.  It is easy to get started with it, but it has very sophisticated features.  It has fewer dark corners than any other language I know.  While there are a few other more popular languages, Python popularity has been steadily growing, especially in the scientific and statistical communities.  But we already have support for three high-level languages, and if there is enough demand, we’ll do more.

Some of our partners prefer to use the lower-level C language interfaces we offer.  That’s fine, too.  We’re not Python zealots (well, maybe, I am).  Python, as a scripting language, isn’t as fast as a compiled language.  For many purposes this does not matter, and Python itself is written in C.  I recently wrote a Python module for TURF analysis.  The computations are simple but computationally explosive, so I was worried that it would be too slow to be useful.   It turned out to be pretty fast because of the way I could use some of Python’s built-in data structures and algorithms.  And the popular numPy and SciPy scientific and numerical libraries are written in C.

Users who would not think of themselves as developers sometimes find that a small Python effort can automate manual work with big time and accuracy improvements.  I got a note recently from a user who said, "I got it to work, and this is FANTASTIC! It will save me a lot of time in my survey analysis work."

Ajay- What are the areas where SPSS is not a good fit for using. What areas suit SPSS software the most compared to other solutions.

Jon- SPSS Statistics, the product,  is not a database.  Our strength is in applying analytical methods to data for model building, prediction, and insight.  Although SPSS Statistics is used in a wide variety of areas, we focus first on people data and think of that first when planning and designing new features.  SPSS Statistics and other SPSS products all work well with databases, and we have solutions for deploying analytics into production systems, but we’re not going to do your payroll.  One thing that was a surprise to me a few years ago is that we have a significant number of users who use SPSS Statistics as a basic reporting product but don’t do any inferential statistics.  They find that they can do customized reporting – often using the Custom Tables module – very quickly.  With Version 17, they can also do fancier and dynamic output formatting without resorting to script writing or manual editing, which is proving very attractive.

Ajay- Are there any plans for SPSS to use Software as a Service Model . Any plans to use advances in remote and cloud computing for SPSS ?

Jon- We are certainly looking at cloud computing.  The biggest challenge is being able to put things in the cloud that will be robust and reliable.

Ajay- What are SPSS’s Asia plans ? Which
country has the maximum penetration of SPSS in terms of usage.

Jon- SPSS, the company, has long been strong in Japan, and Taiwan, and Korea is also strong.  China is increasingly important, of course.  We have a large data center in Singapore.  Although India has a strong, long, history in statistical methodology, it is a much less well-developed market for us.  We have a presence there, but I don’t know the numbers. (Ajay - SPSS has been one of my first experiences in statistical software when I came up with it at my business school in 2001. In India SPSS has been very active with academia licensing and it introduced us to the nice and easy menu driven features of SPSS.)

Biography - Jon earned his Ph. D. from Yale University and taught econometrics and statistics there for 13 years before joining SPSS.

Jon joined the SPSS company in 1983 and worked on many aspects of the very first SPSS DOS product, including writing the first C code that SPSS ever shipped. Among the features he has designed are OMS (the Output Management System), the Visual Bander, Define Variable Properties, ALTER TYPE, Unicode support, and the Date and Time Wizard. Jon is the author of many of the modules on Developer Central. He is an active cyclist and hiker.

Jon Peck blogs on  SPSS Inside-Out.

If you have ever felt frustrated in  knowing business metrics in your or your client organization, negotiated with a host of either legacy applications that don’t talk to each other or good solutions that cost more than the benefit they bring – a young man from India has a solution for you. With a total implementation time range of 1-6 weeks and costs to as low as 10,000 USD for Enterprise WIDE implementation , Dmine promises to shake things up. Here is an interview with the co founder of this startup.

Ajay- Describe your career journey. What advice would you give to  new entrepreneurs in this recession.

Sanjay- Geared with an M. Tech from BITS Pilani, and MBA from Jamnalal Bajaj Institute of Management, Mumbai, in 2000 I teamed up with Praveen Wicliff and  ventured to start our own  Product company with a focus to deliver critical enterprise solutions  and the company has now grown to a strong team of 100 innovative minds. My current mission is to make Icicle a strong leader in business driven Software Products across all segments with the best of the delivery capabilities.
Recession is a trying time for most people and this is one phase which brings out the best in everyone as most of the innovative solutions are floated during this phase. My only suggestion would be to move from emotional connect with customers to direct tangible benefits, stay focussed on cash flow, aim high, set your goals & targets and never give up on any of these. Even in the darkest moments, find the faith to keep going.

Ajay- One more Dashboard Solution. How is dMine different from it’s competitors. What are the principal competitors.

Sanjay- If you just plainly look at dMine you would see that dMine is just another Dashboard solution but what makes dMine different from all other competitive products is its intuitiveness and User friendly features which help even the Business users to use the product most effectively.

dMine is positioned as a product for Business Users and not for IT team. Unlike other Dashboard or BI products dMine can create Dashboards in just 3 easy steps:

1- IT team connects to Data-sources & Creates Business Views

2- Users can create Dashboards & Charts with dMine’s Intuitive interface

3- Users can share Dashboards & schedule Emails in PDF or PPT

The potent combination of best-of-class looking Comprehensive Graphical and Analytical reports, Easier representation and Interpretation of Key Business Data, Integrates data from multiple systems on a single chart and / or Dashboard for real-time Analysis, all these, with minimized IT overheads is a unique proposition from us. See dMine-in-action on  www.dminebi.com and you will know the difference.

Ajay- What is the area where dMine would not be suited for dashboards.Suppose I have data for 200,000 rows x 40 columns – would dMine work for me .

Sanjay-
dMine is positioned as a pure Dashboard product that does not implement a complete BI stack which requires to work on Transactional Data to create cubes and universe.

We look beyond Data warehouses and Datamarts and emphasize on summarized data to deliver key business performance metrics with high focus on Data Visualization.

The idea here is to target the Business Executives who would see these Dashboards and they are not interested in Transactional Data but the overall performance hence the summarized Data.

Here the summarized Data could be in the form of any RDBMS Database, Flat Files, Spreadsheets, Analytics output, Cubes or Universe. We Support almost 16 Database vendors in the market starting from as small as MS Access to as big as DB2.

Ajay- What is the pricing strategy of dMine . Any other products or complements that you are thing about. Name some customer case studies or big wins.

Sanjay- dMine pricing strategy is very simple and is based keeping in mind that the product can be used by customers in the SME/SMB segment or even at the Enterprise level.

Currently we are offering dMine in two forms

• firstly On-Premise and
• second one as a Hosted service.

In an on-premise version, dMine has a Product License fee and per user license fees issued separately.

At additional cost you can have loads of Add-on goodies catering to various needs of the customers. All the cost mentioned above are just one time.

The Hosted version, is on a monthly subscription model where the cost is decided based on the various parameters like the usage of Bandwidth, Server configuration, Disk space etc. Very soon we will be enabled to the Amazon cloud service.

Typically for an on-premise version a smaller implementation just at a corporate level with Dashboard access to only few top executives would cost anywhere between 10,000 USD to 16,000 USD plus the implementation cost which is on actuals plus Applicable taxes.

For larger implementation like for e.g in BFSI segment where you need to roll-out user licenses to all the branch managers in addition to the top executives the total User licenses goes to a few 100 licenses. Usually the implementation period is as low as a week and not more than 6 weeks.

Few of the Customers using dMine are some of the Marketing Analytics companies that use the product for submitting the final report of their Marketing / Customer Analysis to their end customers.

Case Study – Summary

We recently implemented dMine at a leading FM Radio channel to monitor the performance across its radio stations spread all over India.

The client being a major player in the media and communication space has to constantly monitor all their stations for their entire  operations like Revenues from sponsors, Peak and Non-Peak time Inventory & Sales, Market Share & Channel Ranking, P&L, Forecasts and other critical informations.

Currently the client uses multiple Applications for supporting these business functions which capture data in different databases and sources. The MIS-reports were manually created by extracting data from multiple sources in spreadsheets. These spreadsheets are distributed among the management, with a turn around time of about 15 days.

The dMine solution implemented, collates information from multiple data sources like ERP and Sales systems including lots of Spreadsheets. More than 70 Metrics (KPI’s) and Analytics are defined for the Client and now the management has access to these information whenever they require.

All these metrics are identified as critical and are categorized under 5 Dashboards  – Organizational KPI, Financial Dashboard, Sales Dashboard, Market Share
& Metrics Dashboard and Operational Dashboard. The Metrics are parameterized and drill-downs allows the management to get the source of issue/problem rapidly.

The implementation of dMine Business Dashboard product helped the client in effectively monitoring business operations, KPIs, and organizational performance.Making actionable and Real-time information available on-demand for the decision Makers and Operational Managers, has also helped in taking any timely critical business decisions, all these while minimizing IT overhead cost. The short implementation time lines also allows the users to see the benefits quite quickly and achieve the ROI within 1 to 3 months time. The detailed case study is readily available for your reference at our website www.dminebi.com .

Ajay-  Do you read or write blogs. What do you think about the Web 2.0 paradigm for social and community marketing.

Sanjay -I do read and write lots of blogs and am myself a member with quite a few groups that share interest in the virtual community. Web 2.0 provides a platform of many-to-many communications and in its social sense is based on the principles of collaboration & sharing, information & content putting social interaction at heart of it all.

A recent study by Fox Interactive Media reveals that 40% of social network users rely on social media outlets to learn more about brands and products. Whether you’re a freelancer promoting your own brand or part of a company, social media marketing is an essential component of an integrated campaign. If you are looking to startup new business, launch a new product & services or even expand your presence you cannot miss-out on the eMarketing process focusing on three prong strategy i.e. social networking sites, your own website, and the blogosphere these will help empower your brand and positively convey your message.

Ajay – Sanjay Patel is an experienced entrepreneur with Icicle Technologies and the Dmine dashboard is currently winning rave reviews ( see http://www.dminebi.com/ibm-nominates-icicle-as-isv-on-ibm-smart-business-platform/ ) . Here’s wishing luck to Mr Patel for the summarized data dashboard that can be a game changer at www.dminebi.com.

Ajay - Labor arbitrage outsourcing versus virtual teams located globally. What is the SAS Inst position and your opinion on this. What do you feel about the recent debate on HB1 visas and job cuts. How many jobs if at all is SAS planning to cut in 2009-2010.

Anne - SAS is a global company, with customers in more than 100 countries around the world.  We hire employees in these countries to help us better serve our global customers.  Our workforce decisions are based on our business needs.  We also employ virtual teams–the feedback and insights from our global workforce help us improve and develop new products to meet the evolving needs of our customers.  (As someone who works from her home office in Connecticut, I am a fan of virtual teaming!)  We see these approaches as complementary.

The issue of the H-1B visa is a different discussion entirely.  H-1B visas, although capped, permit US employers to bring foreign employees in “specialty occupations” into this country.   The better question, though, is what is necessitating the need for H-1B visas.  We would submit that the reason the U.S. has to look outside its borders for highly qualified technical workers is because we are not producing a sufficient number of workers with the right skill sets to meet U.S. demand.  In turn, that means that our educational system is not producing students interested or qualified to pursue the STEM (science, technology, engineering or mathematics) professions (either at a K-12 or post-secondary level), or developing the workforce improvement programs that may allow workers to pursue these “specialty occupations.”  Further, any discussion about H-1B visas (or any other type of visa) should include a more comprehensive review of our nation’s immigration policies—are they working, are they not working, how or why are they, are we able to limit illegal immigration and if not, why not, etc.

I am not aware of any planned job cuts at SAS.  In fact, I am aware of a few groups which are actively hiring.

Ajay- What open source softwares have SAS Institute worked in the past and it continues to support financially as well as technologically.  Any exciting product releases in 2009-2010 that you can tell us about.

Anne- Open source software provides many options and benefits.  We see many (SAS included) embracing open source for different things.  Our software runs on Linux and we use some open-source tools in development. There are different aspects of open source software in developing SAS software:

-Development with open source tools such as Eclipse, Ant, NAnt, JUnit, etc. to build, test, and package our software

-Using open source software in our products; examples include Apache/Jakarta products such as the Apache Web Server.

-Developing open source software, making changes to an open source codebase, and optionally contributing that source back to the open source project, to adapt an open source project for use in a SAS product or for internal use. Example: Eclipse.

And we plan to do more with open source in the future.  The first step of SAS integrating with R will be shown at SAS Global Forum coming up in DC later this month.  Other announcements for new offerings are also planned at this event. 

Ajay- What do you feel about adopting Software as a service for any of  SAS Institute’s products. Any new initiatives from SAS on the cloud computing front especially in terms of helping customers cut down on hardware costs.

Anne- SAS Solutions OnDemand, the division which oversees the infrastructure and support of all our hosted offerings, is expanding in this rapidly growing market.  SAS Solutions OnDemand Drug Development was our first SaaS offering announced in January.  Additional news on new hosted offerings will be announced at SAS Global Forum later this month.  SAS doesn’t currently offer any external cloud computing options, but we’re actively looking at this area.

Ajay- Which software do you personally find best to write code into and why. Do you miss writing code, if so why ?

Anne- In my current role, I have limited opportunity to write code.  At times, I do miss the logical thought process coding forces you to adopt (to do the job as elegantly as possible).  I had the opportunity to do a long-term assignment at a major financial services company in the UK last year and did get to use some SAS and JMP, including a little JSL (JMP scripting language).  There’s nothing like real-world, noisy, messy data to make you thankful for the power of writing code!  Even though I don’t write code on a regular basis, I am happy to see continued investment in the languages SAS provides—among the most recent, the addition of an algebraic optimization modeling language in our SAS/OR module contained within the SAS language as “PROC OPTMODEL.”

I have great respect for people who invest in learning (or even getting exposure to) more than one language and who appreciate the strengths of different languages for certain tasks and applications.

Ajay- It is great to see passionate people at work on both sides of the open source as well as packaged software teams- and even better for them to collaborate once in a while.Most of our work is based on scientists who came before us (especially in math theory).

Ultimately we are all just students of science anyway.

SAS Global Forum -http://support.sas.com/events/sasglobalforum/2009/

Annual event of SAS language practitioners.SAS language consists of data step and proc steps for input and output thus simplifying syntax for users.

SAS Institute – The leader of analytics software since 1970’s , it grew out of the North Carolina University, and provides jobs to thousands of people. The world’s largest privately held company, admired for it’s huge investments in Research and Development and criticized for its premium price  on packaged software solutions.A recent entrant in corporate users who are willing to support R language.

She was in the news recently with an article by the legendary Ashlee Vance in the Bits Blog of  New York Times http://bits.blogs.nytimes.com/2009/02/16/sas-warms-to-open-source-one-letter-at-a-time/

In the article,  Ms. Milley said, “I think it addresses a niche market for high-end data analysts that want free, readily available code. We have customers who build engines for aircraft. I am happy they are not using freeware when I get on a jet.”

To her credit, Ms. Milley addressed some of the critical comments head-on in a subsequent blog post.

This sparked my curiosity in knowing Anne ,and her perspective more than just a single line quote and here is an interview. This is part 1 of the interview .

Ajay -Describe your career journey , both out of and in SAS Institute. What advice would you give to young high school students to pursue careers in science. Do you think careers in science are as rewarding as other careers.

Anne-

Originally, I wanted to major in international business to leverage my German (which is now waning from lack of use!).  I found the marketing and management classes at the time provided little practical value and happily ended up switching to the college of social science in the economics department, where I was challenged with several quantitative courses and encouraged to always have an analytical perspective.  In school, I was exposed to BASIC, SPSS, SHAZAM, and SAS.  Once I began my thesis (bank failure prediction models and the term structure of interest rates) and started working, it was SAS that served as the best software investment, both in banking (Federal Home Loan Bank of Dallas) and in retail (7-Eleven Corp.).  After 5+ years in Dallas, my husband wanted to move back to New England and SAS happened to be opening an office at the time.  From there, I enjoyed a few years as a pre-sales technical consultant, many years in analytical product management, and most recently in product marketing.  All the while, it has been a great motivating factor to work with so many talented people focused on solving problems, revealing opportunities and doing things better—both within and outside of SAS.

For high school and college students, I urge them to invest in studying some math and science, no matter the career they’re pursuing.  Whether they are interested in banking/finance, medicine and the life sciences, engineering or other fields, courses that will help them explore and analyze data, and come up with new approaches, new solutions, new advances based on a more scientific approach will pay off.

Course work in statistics, operations research, computer science and others will help hone skills for today’s data- and analytics-driven world.  One example of this idea in action:  North Carolina State University’s (NCSU) Institute for Advanced Analytics is seeing a huge increase in interest.  Its first graduating class last year saw higher average salaries than other graduate programs and multiple job offers per graduate.  Why?  Because there is still a huge demand for graduates with the ability to manipulate and analyze data in order to make better, more informed decisions.  I personally think careers in math and science are especially rewarding, but we need many diverse skills to make the world go round )

Ajay- Big corporations versus Startups. Where do you think is the balance between being big in terms of stability and size and being swift and nimble in terms of speed of product roll outs. What are the advantages and disadvantages of being a big corporation in a fast changing technology field.

Anne-

Ever a balancing act, with continuous learning along the way.  The advantage of being big (and privately held) is that you can be more long-term-oriented.  The challenge with fast-changing technology is to know where to best invest.  While others may go to market faster with new capabilities, we seek to provide superior implementations (we invest in ‘R’ (Research) AND ‘D’ (Development), making capabilities available on a number of platforms. 

In today’s economy, I think the big vs. small comparison is becoming less and less relevant.  Big corporations need to be agile and innovative, like their smaller rivals.  And small- to medium-sized businesses (SMBs) need to use the same techniques and technologies as the “big boys.”

First, on the big side, I’ll use an example of which I’m very familiar:  At SAS, a company founded more than 30 years ago as an entrepreneurial venture, we’ve certainly changed over the decades.  SAS started out in a small office with a handful of people.  It’s now a global company with hundreds of offices and thousands of employees around the world.  Yet one thing that has not changed for SAS in all this time:  a laser-like focus on the customer.  This has been the key to SAS’ success and uninterrupted growth .Not really a “secret sauce.” Just a simple yet profound approach: listen carefully to your customers and their changing needs, and innovate, develop and adapt based on these needs.

Of course, being large has its advantages:  we have more ideas from more people, and creativity and innovation knows no borders.  From Sydney to Warsaw, São Paulo to Singapore, Shanghai to Heidelberg, SAS employees work closely with customers to meet their business needs today and in the future.

SAS provides the stability and proven success that businesses look for, particularly in troubled economic times.  Being large and privately held enables SAS to grow when others are cutting back, and continue to invest in R&D at a high rate – 22% of revenues in 2008.

Yet with our annual subscription licensing model, SAS cannot rest on its laurels.  Each year, customers vote with their checkbooks:  if SAS provided them with business benefits, results and a positive ROI, they renew; if not, they can walk away.  Happily for SAS, the overwhelming majority of customers keep coming back.  But the licensing model keeps SAS on its toes, customer-focused, and always listening and innovating based on customer feedback.

As for SMBs, they are rapidly adopting the technologies used by large companies – such as business analytics – to compete in the global economy.  Two examples of this:

BGF Industries is a manufacturer of high-tech fabrics used in jet fighters, bullet-proof vests, movie-theater screens and surfboards, based in Greensboro, NC. BGF turned to SAS business analytics to help it deal with foreign competition.  BGF created a cost-effective, easy-to-use early-warning system that helps it track quality and productivity.  Per BGF, data is now available in minutes instead of hours.  And in the business world, this speed can be the difference between success and failure.  Per Bobby Hull, a BGF systems analyst: “The early-warning system we built with SAS allowed us to go from nothing to everything.  SAS allows us to focus away from clerical tasks to focus on the quality and process side of the job. Because of SAS, we’re never more than three clicks away from finding an answer.”

For Los Angeles-based The Wine House, installing a
SAS-powered
inventory-management system helped it discover nearly $400,000 in “lost” inventory sitting on warehouse shelves.  For an SMB with annual sales of $20 million, that was a major find.  Business analytics helps it to compete with major retail and grocery chains.  Per Bill Knight, owner of The Wine House: “The first day the SAS application was live, we identified approximately 1,000 cases of wine that had not moved in over a year. That’s significant cash tied up in inventory.  We had a huge sale to blow it out, and just in time, because in today’s economy, we would be choking on that inventory.”

So regardless of size, businesses must remain agile, listen to their customers, and use technologies like business analytics to make sense of and derive value from their data – whether on the quality of surfboard covers or the number of cases of Oregon Pinot Noir in stock.

3) SAS Institute has been the de-facto leader in both market volume share as well as market value share in the field of data analytics. What are some of the factors do you think have contributed to this enduring success. What have been the principal challengers over the years.(Any comments on the challenge from SAS language software WPS please ??)

At SAS, we seek to provide a complete environment for analytics—from data collection, data manipulation, data exploration, data analysis, deployment of results – and the means to manage that whole process.  Competition comes in many forms and it pushes us to keep delivering value.  For me, one thing that sets SAS apart from other vendors is that we care so deeply about the quality of results.  Our Technical Support, Education and consulting services organizations really do partner with customers to help them achieve the best results.  That kind of commitment is deep in the DNA of SAS’ culture.

The good thing about competition is that it forces you to re-examine your value proposition and rethink your business strategy.  Customers value attributes of their analytics infrastructure in varying degrees— speed, quality, support, flexibility, ease of migration, backward and forward compatibility, etc.  Often there are options to trump any one or a subset of these and when that aligns with the customers’ priorities of what they value, they will vote with their pocketbooks.  For some customers with tight batch-processing windows, speed trumps everything.  In tests conducted by Merrill Consultants, an MXG program running on WPS runs significantly longer, consumes more CPU time and requires more memory than the same MXG program hosted on its native SAS platform.

While it’s easy to get caught up in fast-changing technology, one has to also consider history.  Some programming languages come and go; others have stood the test of time.  Even the use of different flavors of analysis ebbs and flows.  For instance, when data mining was all the rage almost a decade ago, many asked the very good question, “Why so much excitement about analyzing so much opportunistic data when design of experiments offers so much more?”  Finally, experimental design is being more readily adopted in areas like marketing.

At the end of the day, innovation is the only sustainable competitive advantage.  As noted above in question 2, SAS has remained firmly committed to customer-driven innovation.  And SAS has “stuck to its knitting” with respect to analytics.  A while back, SAS used to stand for “Statistical Analysis System.” If not literally, then philosophically, Analytics remains our middle name.

(Ajay- to be continued)

Continuing with the Slumdog Millionaire celebrations in India , we have an interview here at DecisionStats with an up and coming intense Indian film maker. Nitin Dash is a creative film maker based out of India. He has created movies like the science fiction movie “ Formula 69” , short videos like “500” (see below) . He gave up a corporate career after 5 tears of corporate experience and after studying at the renowned  Indian Institute of Management, Lucknow to pursue his creative side. Here in a candid interview , Nitin discusses the things that motivate him and passes some tips for home movie making. Coming from a person that has 200,000 plus views on his YouTube video , it is nifty and useful advice.

Ajay – What has been your educational and career journey so far . 

Nitin- Finished my MBA from IIML in 2000. Worked for 5 years in the corporate media sector.. Did a short 6- month course in filmmaking from New York in  2005. Started my own film production company  Filmkaar Productions.. www.filmkaar.com

Ajay – What inspires your art. What are the key things that made you decide to take a leap into movie direction from the corporate world.

Nitin – I find inspiration from people around me. A common man , his life and the simple conflicts and challenges that make it interesting..
I felt that the corporate world was stifling my creativity and the work was very operational and mundane. I had some friends from Jamia mass comm.. school. We got together and started making short films over the weekend. After a few months I decided that film making is my calling in life.

Ajay – I have a Sony hand camera and I would like to be Steven Spielberg while shooting my son’s videos. Comment please. Give me 5 bullet points or tips.

Nitin -  The following links would help you.

www.video101course.com
www.cybercollege.com

Learn windows moviemaker. It is a very simple to learn and easy to use software and already installed on PC’s with windows xp / vista

Ajay- How effective do you think is viral marketing . What paradigm changes do you think have Web 2.0 ,blogs, YouTube brought about in the traditional content business.

Nitin- Viral marketing is very effective, but the content has to be right. Getting a good
content that turns into a viral is very difficult and most of the times unpredictable. Web 2.0 has given access to people to express their creativity and share it with the world.

Ajay-What do you think about the  casting couch as a director ? 1 line comment please

Nitin- It’s unofficial term for networking in the film industry.

Ajay- What has been your most successful movie- short film. Which short movie do you like the most and why ?

Nitin-http://www.youtube.com/watch?v=35u0J4p26Fg
A magical tale about a young boy who finds a solution for Global Warming from a monk in the mountains. I like the simplicity of the story and the beautiful location it is shot in.

Ajay- Well there is no telling when Nitin would bring home an Oscar , but you can preview a short 2 minute video by him. Its called 500 and describes how different people would spend Rs 500 if they found it. Simple and powerful on the ways money moves us in different ways-and the social disparities in India shining.

 Filmkaar Productions has been set up to promote thought provoking cinema.To create entertaining films that are socially relevant. Engaging films that can transform minds.Our mission is to make ‘Extraordinary films with Ordinary people’. Visit them at www.filmkaar.com

Sevana is a Finland based company which creates excellent software and analytics  products and their latest release is their automated audio quality product. Existing releases have been a shopping cart analyzer which does wonderful automated market basket analysis.

Ajay – What has been your career journey so far ? What advice would you
give to a fresh science graduate entering the market in today’s
recession .

Endre - About my career journey 

After receiving an MSc in Electronic Engineering my first job was maintenance of the Soviet "clone" of an IBM/360 computer (I still remember some of the Russian language terminology).While doing post graduate studies (got something that would be called today Tech.Lic. in Data Communication) I was offered a job by one of the professors in a research institute. Through the research institute I got a chance to work on a nuclear powerplant simulator in Finland as a Hungarian ex-pat (important, because Tsernobyl happened in the meanwhile).

I specified and implemented the mainframe side of the communication between a VAX/VMS mainframe and several PDP’s  (I’m still proud that later on someone who saw my part of the system, written in 1986, said that it was object-oriented, but the language was Fortran

One of the jobs enjoyed most was at Fiskars Power electronics. I could design the Hardware and write all software for a microcontroller-based intelligent display of a UPS (uninterruptable (or unpredictable?) power supply), which communicated with the UPS via the power line (around 1988-89).

Then 6 years at Nokia and 5 years at Nokia Research Center, where I got more familiar with object-orientation.A brief stop at Rational, followed by lecturing at the Helsinki Technical University for about 3 years (concurrent programming; UML-related topics). Somewhere in the meantime a (or rather THE) company has been founded, where I still work.

Here is the answer to the "advice" part

My advice would be – if we were speaking of a bright graduate – that his decision to start establishing contacts with potential employers during his studies and to lay down the foundations of his professional network was very wise, and now he should start using his contacts.

Finding a good position on the labor market, or a place on the IT market with a product or idea involves a certain amount of luck but also planning and conscious self management, the sooner career starters realize this the better.

Ajay - What are the key things that you have worked with in terms of technologies.

Endre- To my opinion it’s always a matter of people rather than anything else,
because people create technologies and people use technologies.

I believe that the key technologies we worked with are the way our company is organized and managed, the way our employees treat working with us and of course that state-of-the-art products (no matter what actual technology we have in mind: C, .NET, Delphi, PHP, Java etc), which our employees develop for our customers.

Two major examples are existing product providing automated audio quality measurement and analysis and the tool to mine and manage association rules in high data volumes that we expect to release QI 2009. Both are unique on the market as technology/science wise as well as functionality wise.

Ajay- What is the most creative product that has been released or is going to be released by your company.

Endre- I would mention the same two analytical products:

Automated audio/voice quality estimation is already released and we are searching and negotiating with companies to partner on its dissemination and integration to voice quality and quality of service test solutions.

All information about scientific approach, technology, tests and benefits is available from our web site (www.sevana.fi) partly freely and partly under NDA.We also put big hopes for the association rules mining system, which we develop trying to take into account needs of statisticians and marketing/sales analysts as well as typical demands in various industries: retail, wholesale, maintenance. I would like to give special thanks to Mr. Ajay Ohri whom we were consulting with about the features of such product and its market applications and demand. ( Ajay- Pleasure is mine)

Ajay-  Outsourcing has taken off really well in Poland and Romania. What
are the best known success stories of outsourcing that you can tell
of.What are the best known success stories of outsourcing that you can tell of.

Endre- Well, outsourcing may have different faces – it can be a big success and a
big failure or even a failure with a face of success. I believe that success story for software outsourcing is any company that has established a well operating and profitable company in any country, where doing software outsourcing makes sense.

I also believe that we have a good concept for software outsourcing projects as well, providing onshore software development at offshore prices in Finland.

We have our own know-how in order to make it possible.

Ajay- What do you think about the open source versus proprietary software debate. What is scenario in your local market ( across parts of the country ) regarding this.

Endre- Open source gives the freedom to the “evolution” of applications and services.

It can spare you from reinventing the wheel. I forgot the source, but some famous computer scientist said something like: if programmers read more they would have to write less (code)One can argue that in case of open source one doesn’t easily find a bug-fix if her/his problem is not "mainstream".

However, even in proprietary software the vendor has priorities (often market-driven) and if your wallet is not thick enough and you are at the end of the list you’ll have to wait. And fixing, making a workaround, on your own is much more difficult.

Ajay – What are the intellectual property rights conditions as well as language facilities for Russian software companies ? What is the best way to contact local Russian companies for a software contract.

Endre- Protecting intellectual property rights is a reasonable issue in Russia and a lot of effort is put to improve the situation by the government and business, however I believe that the same challenges can be found in any other country: if your IPRs are broken for instance by your outsourcing company, would you really be able to afford court trial? I am sure not every company would be able to afford it no matter where we have IPR violation: in Russia, Romania, Poland or India.

I think the best way is to try to contact individuals first, because in Russia for instance there are a lot of highly qualified people who would rather try to establish their own
business than trying to be highly recognized by local outsourcing companies. We’ll be happy to assist in providing connections to the Russian software compan
ies and individuals.

Disclaimer- Ajay- I advise Sevana on Web 2.0 initiatives .See more on their products at http://wordpress.sevana.fi/ and http://sevana.fi

Phil Rack is the creator of a Bridge to R and SAS Bridge to R which enables both WPS and SAS softwares to connect to R. He is also a WPS Reseller. WPS is a base SAS equivalent that can take in SAS code , SAS datasets, write SAS code, and create SAS datasets ( and also create its own format)- at the cost of 660 $ a license ( and almost one tenth of a SAS Institute installation on network servers). Having worked in SAS language and analytics consulting for almost 26 years ,Phil runs www.minequest.com besides running the SAS Consultants network that mentors analytics consultant globally ( I am an ex- member )

Ajay- What has been your career journey. What advice would you give to someone entering a science career after high school?

Phil- I started out consulting full-time in 1983. I left an analytics job with McMillan-McGraw-Hill Publishing because I didn’t believe the company was investing in BI tools and training as it should. That was pretty early in terms of when BI was becoming important.

Many companies at that point saw BI as only two things.

(a) Ability to forecast sales and

(b) ad-hoc reporting with sums/totals and percentages.

It was obvious to me that I had to make a change to do the kind of work I wanted to do. In terms of training, I was formally trained as a demographer and did my graduate studies at Ohio State so I received a pretty good dose of quantitative subject matter as well as a unique perspective on the social implications of markets and geography. If I had to do it over again, I would probably take more course work in the subject area of the "Family."  I’m always amazed how many times the work I do in banking and finance revolves around the family lifecycle.

Ajay- What has been the biggest project success you have seen in your consulting practice?

Phil- This goes back three years to a project where I was working on Basel II compliance with a commercial banking client that I just loved working with.

A few months into the engagement, they pulled me aside and asked me to put together an automotive portfolio stress test for them. This bank had very large loan exposures to the auto market for second and third tier suppliers to the Big Three as well as international auto manufacturers.

The Risk Management group and I sat down for a couple of days and pulled together a project plan and an outline of what we needed to be able implement a dynamic Auto Risk Stress Test Model for this portfolio. The software used was SAS/Base and Excel and the program allowed us to modify 50 to 60 parameters to model different scenarios. All together, it took perhaps three weeks to implement and it was amazingly indicative of the fall out of the auto industry as well as foreshadowing some of the financial carnage in South Eastern Michigan such as lower property values and unemployment.

Ajay- "It is not what you know, it is whom you know." Comment please as an SAS consultant.

Phil- In terms of my business, 80-90% of the work I do is either based on prior work that I’ve done for that company or through referrals.  If you want to have a successful consulting career, you really have to pay attention to developing your network. I’ve taken advantage of social gatherings such as charity events and other social mixers to try to extend my network. I hand out a lot of business cards every year. Formal organizations exist here in Columbus, Ohio such as TechColumbus.org that is a dynamite organization that helps small tech businesses in the area of networking, financing, access to different hardware platforms for testing, etc…  I have mixed emotions about the value of some social networks however.

I see so many individuals on LinkedIn that have 5,000 connections that I have to wonder what it is these folks really do. Who has the time to read all the updates and postings for 5,000 people and still be able to get work out the door? ( Note from Ajay- I have 6300 connections on LinkedIn . Ouch !!)

Ajay- What motivated you to write the SAS to R and WPS to R bridges? (Which IS your favorite analytical tool, since you are active in all three?)

Phil- It started out as a "proof-of-concept" exercise and it’s just keeps growing. The WPS to R Bridge is a piece of software that I wrote originally for WPS users to access R from within the WPS Workbench. For those who are unfamiliar with WPS, it’s a SAS/Base alternative that is extremely compatible with your existing SAS/Base software and your code is just plug-and-play. WPS doesn’t have the statistical capabilities of SAS such as SAS/STAT, ETS, OR, etc… so the idea was to write a bridge so that WPS users wouldn’t have to learn a new GUI/IDE to use R. The Bridge gives WPS users access to R graphics as well as any of the R statistical libraries but it has the advantage of the superior data handling of the SAS language. One of the new features is the ability of the WPS to R Bridge to run R programs in parallel. Depending on your hardware, you can easily run six to a dozen R programs simultaneously and collect the R listing and log files back into the WPS listing and log in the order you submitted the programs.

I did write a Bridge to R for SAS users but very few SAS users have expressed interest in it. I suppose that SAS users are happy enough paying the fat licensing fees to SAS that it just doesn’t matter to them. I have to say, my favorite tool at the moment is WPS. I find the interface/workbench to be so superior to what SAS has to offer that I now find myself writing code in WPS and then taking it over to SAS if that’s what the client requires.

Ajay- What do you think about internet based delivery and social networking including communities and lists changing the software product cycle?

Phil- This somewhat goes back to question #3 in terms of communities. I think it has its value as a place to share your concerns and find answers to difficult programming issues. Now, Internet delivery and cloud computing I find very interesting. I think there’s some strong advantages to using the cloud to provide services to your clients. If you look at the SAS pricing model, they really take it to you financially if you want to use your license to be a DSP (data service provider) or put your code on an intranet/internet. For some reason, SAS is just hostile when it comes to small and medium sized businesses. Companies like World Programming who license WPS have a much more realistic idea of licensing in that you can expose your WPS license to your intranet/internet and not have to pay 10x the fees that SAS charges. WPS doesn’t charge additional fees for those who are DSP’s either and there are quite a few of them in the Pharma domain.

Beyond security challenges associated with cloud computing, I think SaaS that provides analytical services such as high performance forecasting and name and address cleanup and verification are ripe for the picking. One other issue I see with cloud computing is when you have tens of gigs of data that you have to move from your desktop or server to the cloud. The infrastructure just isn’t fast enough, or let’s say reasonably priced, to allow for moving this amount of data to really scale well.

Ajay- How does MineQuest intend to influence the analytical software paradigm?

Phil- I think the role for MineQuest in the next few years is twofold.

We’ll keep offering services to banks and other financial service firms in the area of Operational Risk and SAS programming.

The other area is to help these large financial service companies realize that they can save millions of dollars by moving their SAS Server licenses to WPS. This
also allows the smaller businesses who have steered away from SAS software because of cost to begin using WPS and not take such a big financial hit. I find it exciting to think how this will also open the job market for the thousands of SAS programmers out there already.

The BI battles are taking place on the desktop and Windows Servers and MineQuest has invested a lot of time and effort in creating macro libraries to help these organizations migrate their code to WPS and access R for advanced statistical capabilities.

We believe that the bread and butter software for almost any financial organization in the BI realm ultimately revolves around the SAS language for reporting, summarization and disbursement of data and we plan to continue to serve that market.

 About Minequest -

MineQuest has been providing SAS Consulting and Programming Services for more than 25 years. Our associates and employees are expert SAS programmers and specialize in the Banking and Financial Industries. Our staff has expertise in such areas as Market Analytics, ETL and Reporting Systems, Fraud Detection, and Credit Risk and Operational Risk segments. Validating Operational Risk models using SAS, in support of the Basel II Capital Framework is one of our specialties. We have real world experience developing SAS software to test and validate Credit and Operational Risk Systems like Fair Isaac’s Blaze Advisor which is one of our areas with subject matter expertise.

MineQuest, LLC

SAS & WPS Consulting and WPS Reseller

Tel: (614) 457-3714

Web: www.MineQuest.com

Blog: www.MineQuest.com/WordPress

( Ajay –

SAS language uses mainly Procs and Data step for output and input.Base SAS is a product copyrighted by the SAS Institute (www.sas.com) .SAS Institute has been leading the analytics world since the 70’s.WPS is copyright of World Programming Company (WPC) (www.teamwpc.co.uk/products/wps ) )

Ajay- What are the traditional rivals to scoring solutions offered by you. How does ADAPA compare to each of them. Case Study- Assume I have 50000 leads daily on a Car buying website. How would ADAPA help me in scoring the model ( created say by KXEN or , R or,SAS, or SPSS).What would my approximate cost advantages be if I intend to mail say the top 5 deciles everyday.

Michael- Some of the traditional scoring solutions used today are based on SAS, in-database scoring like Oracle, MS SQL Server, or very often even custom code.  ADAPA is able to import the models from all tools that support the PMML standard, so any of the above tools, open source or commercial, could serve as an excellent development environment.

The key differentiators for ADAPA are simple and focus on cost-effective deployment:

1) Open Standards – PMML & SOA:

Freedom to select best-of-breed development tools without being locked into a specific vendor;  integrate easily with other systems.

2) SaaS-based Cloud Computing:

Delivers a quantum leap in cost-effectiveness without compromising on scalability.

In your example, I assume that you’d be able to score your 50,000 leads in one hour using one ADAPA engine on Amazon.  Therefore, you could choose to either spend US$100,000 or more on hardware, software, maintenance, IT services, etc., write a project proposal, get it approved by management, and be ready to score your model in 6-12 months…

OR, you could use ADAPA at something around US$1-$2 per day for the scenario above and get started today!  To get my point across here, I am of course simplifying the scenario a little bit, but in essence these are your choices.

Sounds too good to be true?  We often get this response, so please feel free to contact us today [http://www.zementis.com/contact.htm] and we will be happy show you how easy it can be to deploy predictive models with ADAPA!

Ajay- The ADAPA solution seems to save money on both hardware and software costs. Comment please. Also any benchmarking tests that you have done on a traditional scoring configuration system versus ADAPA.

Michael-Absolutely, the ADAPA Predictive Analytics Edition [http://www.zementis.com/predictive_analytics_edition.htm] on Amazon’s cloud computing infrastructure (Amazon EC2) eliminates the upfront investment in hardware and software.  It is a true Software as a Service (SaaS) offering on Amazon EC2 [http://www.zementis.com/howtobuy.htm] whereby users only pay for the actual machine time starting at less than US$1 per machine hour.  The ADAPA SaaS model is extremely dynamic, e.g., a user is able to select an instance type most appropriate for the job at hand (small, large, x-large) or launch one or even 100 instances within minutes.

In addition to the above savings in hardware/software, ADAPA also cuts the time-to-market for new models (priceless!) which adds to business agility, something truly critical for the current economic climate.

Regarding a benchmark comparison, it really depends on what is most important to the business.  Business agility, time-to-market, open standards for integration, or pure scoring performance?  ADAPA addresses all of the above.  At its core, it is a highly scalable scoring engine which is able to process thousands of transactions per second.  To tackle even the largest problems, it is easy to scale ADAPA via more CPUs, clustering, or parallel execution on multiple independent instances. 

Need to score lots of data once a month which would take 100 hours on one computer?  Simply launch 10 instances and complete the job in 10 hours over night.  No extra software licenses, no extra hardware to buy — that’s capacity truly on-demand, whenever needed, and cost-effective.

Ajay- What has been your vision for Zementis. What exciting products are we going to see from it next.

Michael - Our vision at Zementis [http://www.zementis.com] has been to make it easier for users to leverage analytics.  The primary focus of our products is on the deployment side, i.e., how to integrate predictive models into the business process and leverage them in real-time.  The complexity of deployment and the cost associated with it has been the main hurdle for a more widespread adoption of predictive analytics. 

Adhering to open standards like the Predictive Model Markup Language (PMML) [http://www.dmg.org/] and SOA-based integration, our ADAPA engine [http://www.zementis.com/products.htm] paves the way for new use cases of predictive analytics — wherever a painless, fast production deployment of models is critical or where the cost of real-time scoring has been prohibitive to date.

We will continue to contribute to the R/PMML export package [http://www.zementis.com/pmml_exporters.htm] and extend our free PMML converter [http://www.zementis.com/pmml_converters.htm] to support the adoption of the standard.  We believe that the analytics industry will benefit from open standards and we are just beginning to grasp what data-driven decision technology can do for us.  Without giving away much of our roadmap, please stay tuned for more exciting products that will make it easier for businesses to leverage the power of predictive analytics!

Ajay- Any India or Asia specific plans for the Zementis.

Michael-Zementis already serves customers in the Asia/Pacific region from its office in Hong Kong.  We expect rapid growth for predictive analytics in the region and we think our cost-effective SaaS solution on Amazon EC2 will be of great service to this market.  I could see various analytics outsourcing and consulting firms benefit from using ADAPA as their primary delivery mechanism to provide clients with predictive  models that are ready to be executed on-demand.

Ajay-What do you believe be the biggest challenges for analytics in 2009. What are the biggest opportunities.

Michael-The biggest challenge for analytics will most likely be the reduction in technology spending in a deep, global recession.  At the same time, companies must take advantage of analytics to cut cost, optimize processes, and to become more competitive.  Therefore, the biggest opportunity for analytics will be in the SaaS field, enabling clients to employ analytics without upfront capital expenditures.

Ajay – What made you choose a career in science. Describe your journey so far.What would your advice be to young science graduates in this recessionary times.

Michael- As a physicist, my research focused on neural networks and intelligent systems.  Predictive analytics is a grea
t way for me to stay close to science while applying such complex algorithms to solve real business problems.  Even in a recession, there is always a need for good people with the desire to excel in their profession.  Starting your career, I’d say the best way is to remain broad in expertise rather than being too specialized on one particular industry or proficient in a single analytics tool.  A good foundation of math and computer science, combined with curiosity in how to apply analytics to specific business problems will provide opportunities, even in the current economic climate.

About Zementis

Zementis, Inc. is a software company focused on predictive analytics and advanced Enterprise Decision Management technology. We combine science and software to create superior business and industrial solutions for our clients. Our scientific expertise includes statistical algorithms, machine learning, neural networks, and intelligent systems and our scientists have a proven record in producing effective predictive models to extract hidden patterns from a variety of data types. It is complemented by our product offering ADAPA®, a decision engine framework for real-time execution of predictive models and rules. For more information please visit www.zementis.com

Ajay-If you have a lot of data ( GB’s and GB’s) , an existing model ( in SAS,SPSS,R) which you converted to PMML, and it is time for you to choose between spending more money to upgrade your hardware, renew your software licenses  then instead take a look at the ADAPA from www.zementis.com and score models as low as 1$ per hour. Check it out ( test and control !!)

Do you have any additional queries from Michael ? Use the comments page to ask….

Updated  –I  finally got my hardback copy of the R for SAS and SPSS users . Digital copies are one thing, but a paper book is really beautiful .I had written an article on R ( with some mild sarcasm on some other softwares that are mildly more expensive) at Smart Data Collective. That created around 711 views of that article, ( my website got X00 hits that day, which is a personal best ,ehmm

It also inspired Sandro, a terrific data miner from Switzerland and a PhD to write an article called 5 reasons R is good for you, which can be accessed here http://smartdatacollective.com/Home/15756 and http://dataminingresearch.blogspot.com/2009/01/top-5-reasons-r-is-good-for-you.html

The story of how I wrote that Top Ten R article is also amusing – mentioned here by Jerry who creates terrific communities for content , all extremely digital and informative , readable here -http://www.socialmediatoday.com/SMC/67268

Now the reason I originally became involved with R, was because I couldn’t afford SAS and SPSS on my own computer after years of getting companies to pick up the tab. A question on the R help list led me to Bob Muenchen , who had written a short guidebook on R for SAS and SPSS users, and was then finishing his book. The following article is interesting given that it was done almost 3-4 months back yet some themes and events seemed to recur exactly as Bob mentioned them. I still bounce between Bob’s book and the Rattle guide for R programming but I am getting there !!!

Note-Robert Muenchen (pronounced Min’-chen) is the author of the famous R for SAS and SPSS users, and his book is an extensive tutorial on anyone wanting to learn either SAS,SPSS,or R or even to migrate from one platform to another. In an exclusive interview Bob agreed to answer some questions on the book , and on students planning to enter science careers.

What made you write the R For SAS and SPSS users?

The book-

A few years ago, all my colleagues seemed to be suddenly talking about R. Had I tried it? What did I think? Wasn’t it amazing? I searched around for a review and found an article by Patrick Burns, "R Relative to Statistics Packages" which is posted on the UCLA site (http://www.ats.ucla.edu/stat/technicalreports/). That article pointed out the many advantages of R and in it Burns claimed that knowing a standard statistics package interfered with learning R. That article really got my interest up. Pat’s article was a rejoinder to "Strategically using General Purpose Statistics Packages: A Look at Stata, SAS and SPSS" by Michael Mitchell, then the manager of statistical consulting at UCLA (it’s at that same site). In it he said little about R, other than he had "enormous difficulties" learning it that he had especially found the documentation lacking.

I dove in and started learning R. It was incredibly hard work, most of which was caused by my expectations of how I thought it ought to work. I did have a lot to "unlearn" but once I figured a certain step out, I could see that explaining it to another SAS or SPSS user would be relatively easy. I started keeping notes on these differences for myself initially. I finally posted them on the Internet as the first version of R for SAS and SPSS Users. It was only 80 pages and much of its explanation was in the form of extensive R program comments. I provided 27 example programs, each done in SAS, SPSS and R. A person could see how they differed, topic by topic. When a person ran the sections of the R programs and read all the comments, he or she would learn how R worked.

A web page counter on that document showed it was getting about 10,000 hits a month. That translates into about 300 users, paging back and forth through the document. An editor from Springer emailed me to ask if I could make it a book. I said it might be 150 pages when I wrote out the prose to replace all the comments. It turned out to be 480 pages!

What are the salient points in this book ?

The main point is that having R taught to you using terms you already know will make R much easier to learn. SAS and SPSS concepts are used in the body of the book as well as the table of contents, the index and even the glossary. For example, the table of contents has an entry for "Value Labels or Formats" even though R uses neither of those terms as SPSS and SAS do, respectively. The index alone took over 80 hours to compile because it is important for people to be able to look up things like "length" as both a SAS statement and as an R function. The glossary defines R terms using SAS/SPSS jargon and then again using proper R definitions.

SAS and SPSS each have five main parts: 1) commands to read and manage data, 2) procedures for statistics & graphics, 3) output management systems that allow you to use output as input to other analyses, 4) a macro language to automate the above steps and finally 5) a matrix language to help you extend the packages. All five of these parts use different statements and rules that do not apply to the others. Due to the complexity of all this, many SAS and SPSS users never get past the first two parts.

R instead has all these functions unified into a common single structure. That makes it much more flexible and powerful. This claim may seem to be a matter of opinion, but the evidence to back it up comes from the companies themselves. The developers at SAS Institute and SPSS Inc. don’t write their procedures in their own languages, R developers do.

How do you think R will impact the statistical software vendors?

With more statistical procedures than any other package, and its free price, some people think R will put many of the proprietary vendors out of business. R is a tsunami coming at the vendors and how they respond will determine their future. Take SPSS Inc. for example. They have written an excellent interface to R that lets you transfer your data back and forth, letting you run R functions in the middle of your SPSS programs. I show how to use it in my book. Starting with SPSS 17, you can also add R functions to the SPSS menus. This is particularly important because most SPSS users prefer to use menus. The company itself is adding menus to R functions, letting them rapidly expand SPSS’ capabilities at very little expense. They saw the R tsunami coming and they hopped on a surfboard to make the most of it. I think this attitude will help them thrive in the future.

SAS Institute so far as been ignoring R. That means if you need to use an analytic method that is only available in R, you must learn much more R than an SPSS user would. Once you have done that, you might be much more likely to switch over completely to R. Colleagues inside SAS Institute tell me they are debating whether they should follow SPSS’ lead and write a link to R. T
his has already been done by MineQuest, LLC (see http://www.minequest.com/Products.html ) with their amusingly named, "A bridge to R" product (playing off "A Bridge Too Far.")

Statistica is officially supporting R. You can read about the details at (http://www.statsoft.com/industries/Rlanguage.htm) . Statacorp has not supported R in Stata yet, although a user, Roger Newson, has written an R interface to it (http://ideas.repec.org/c/boc/bocode/s456847.html).

The company with the most to lose are the makers of S-PLUS. That was Insightful Corp. until they were recently bought out by Tibco. Since R is an implementation of the S language, S-PLUS could be hit pretty hard. On the other hand, they do have functions that handle "big data" so there is a chance that people will develop programs in R, run out of memory and then end up porting them to S-PLUS. S-PLUS also has a more comprehensive graphical user interface than R does, giving them an advantage. However, XL-Solutions Corp. has their new R-PLUS version that adds a slick GUI to R (http://www.experience-rplus.com/). There could be a rocky road ahead for S-PLUS. IBM faced a similar dilemma when computing hardware started becoming commodities. They prospered by making up the difference with service income. Perhaps Tibco can too.

Do you have special discounts for students?

My original version of R for SAS and SPSS Users is still online at http://RforSASandSPSSusers.com so students can get it there for free. The book version has a small market that is mostly students so pricing was set with that in mind.

What made you choose a career in Science and what have been the reasons for your success in it.

I started out as an accounting major. I was lucky enough to have had two years of bookkeeping in high school, and I worked part-time in the accounting department of ServiceMaster Industries for several years. I got to fill in for whoever was on vacation, so I got a broad range of accounting experience. I also got my first experience with statistics by helping the auditors. We took a stratified sample of transactions. With transactions divided into segments by their value, and sample a greater proportion as the value increased. For the most expensive transactions, we examined them all. My job was to be the "gofer" who collected all the invoices, checks, etc. to prove that the transactions were real. For a kid in high school, that was great fun!

By the time I was a freshman at Bradley University, I became excited by three new areas: mathematics, computing and psychology. I got to work in a lab at the Peoria Addictions Research Institute, studying addiction in rats and the parts of the brain that were involved. I wrote a simple stat package in FORTRAN to analyze data. After getting my B.A. in psychology, I worked on a PhD in Educational Psychology at Arizona State University. I loved that field and did well, but the job market for professors in that field was horrible at the time. So I transferred to a PhD program in Industrial/Organizational Psychology at The University of Tennessee. It turned out that I did not really care for that area at all, and I spent much of my time studying computing and calculus. My assistantship was with the Department of Statistics. By the time my first year was up, I transferred to statistics. At the time the department lacked a PhD program, so after four years of grad school I stopped with an M.S. in Statistics and got a job as a computing consultant helping people with their SAS, SPSS and STATGRAPHICS programs. Later I was able to expand that role, creating a full-fledged statistical consulting center in partnership with the Department of Statistics. Ongoing funding cuts have been chipping away at that concept though.

What made me a success? I love my job! I get to work with a lot of smart scientists and their grad students, expanding scientific knowledge. What could be better?

Science is boring, and not well paying career compared to being a lawyer or a sales job. People think you are a nerd. Please comment based on your experiences.

Science is constantly making new discoveries. That’s not boring! An area that most people can relate to is medicine. When we finish a study that shows a new treatment is better than an old one, our efforts will help thousands of people. In one study we compared a new, very expensive anti-nausea drug to an old one that was quite cheap. The pharmaceutical company claimed the new drug was better of course, but our study showed that it was not. That ended up helping to control health care costs that we all see escalating rapidly.

Another study found for the first time, a measure that could predict how well a hearing aid would help a person. Now, it’s easy to measure a hearing aid and see that it is doing what it is supposed to do, but a huge proportion of people who buy them don’t like them and stop wearing them after a brief period. Scientists tried for decades to predict which people would not be good candidates for hearing aids. A very sharp scientist at UT, Anna Nabelek, came up with the concept of Acceptable Noise Level. We measured how much background noise people were willing to tolerate before trying a hearing aid. That allowed us to develop a model that could predict well for the first time if someone should bother spending up to $5,000 for hearing aids. For retired people on a fixed income, that was an important finding. An audiology journal devoted an entire issue to the work.

It’s true that you can make more money in many other fields. But the excitement of discovery and the feeling that I’m helping to extend science very satisfying and well worth the lower salary. Plus, having a job in science means you will never have a chance to get bored!

What is your view on Rice University’s initiatives to create open source textbooks at http://cnx.org/ .

I think this is a really good idea. One of my favorite statistics books is Statnotes: Topics in Multivariate Analysis, by G David Garson. You can read it for free at http://www2.chass.ncsu.edu/garson/pa765/statnote.htm .

Universities pay professors to spend their time doing research, which must be published to get credit. So why not pay professors to write text books too? There have been probably hundreds of introductory books in every imaginable field. They cannot all make it in the marketplace so when they drop out of publication, why not make them available for free? I still have my old Introductory Statistics textbook from 30 years ago and the material is still good. It may be missing a few modern things like boxplots, but it would not take much effort to bring it up to date.

I’m also a huge fan of Project Gutenburg (http://www.archive.org/details/gutenberg). That is a collection of over 20,000 books, articles, etc. available there for free download. My wife does volunteer project management and post-processing with Distributed Proofreaders (http://www.pgdp.net/) which supplies books for Gutenburg.

What are your vie

ws on students uploading scanned copies of books to torrent sharing web sites because of expensive books.

The cost of textbooks has gotten out of hand. I think students should pressure universities and professors to consider cheaper alternatives. However scanning books putting them up on web sites isn’t sharing, it’s stealing. I put in most of my weekends and nights for 2 ½ years on my book that will be lucky to sell a few thousand copies. That works out to pennies per hour. Seeing it scanned in would be quite depressing.

When is the book coming out ? What is taking so long ?

We ran into problems when the book was translated from Microsoft Word to LaTeX. The translator program did not anticipate that an index would already be in place. That resulted in 2-3 errors per page. We’re working through that and should finally get it printed in early October.

Biography

Robert A. Muenchen is a consulting statistician with 28 years of experience. He is currently the manager of the Statistical Consulting Center at the University of Tennessee. He holds a B.A. in Psychology and an M.S. in Statistics. Bob has conducted research for a variety of public and private organizations and has assisted on more than 1,000 graduate theses and dissertations. He has coauthored over 40 articles published in scientific journals and conference proceedings. Bob has served on the advisory boards of SPSS Inc., the Statistical Graphics Corporation and PC Week Magazine. His suggested improvements have been incorporated into SAS, SPSS, JMP, STATGRAPHICS and several R packages. His research interests include statistical computing, data graphics and visualization,text analysis, data mining, psychometrics and resampling.

Ajay-He is also a very modest and great human being.

http://www.amazon.com/SAS-SPSS-Users-Statistics-Computing/dp/0387094172/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1217456813&sr=8-1

I first talked about KXEN,the automated modeling software  in this post http://www.decisionstats.com/2008/12/automating-regression-models-kxen/

So I asked Roger Haddad ,its founder and CEO if he could give an email interview and Roger being the great guy he is , both remembered me from user analyst days as well worked in the holidays to give this interview. Before founding KXEN, Mr. Roger Haddad was president of Azlan France, the first network distributor in France. Under his management, sales revenues increased 850 percent in four years.In 1977, Mr. Haddad founded Metrologie International, a leading software and hardware provider, which went public on the Paris Stock Exchange in 1985. When Mr. Haddad left in 1991, Metrologie had 4,500 employees, $900M in revenue and subsidiaries in 13 European countries.Mr. Haddad holds a master’s degree in electrical metrology from George Washington University and a bachelor’s degree in electrical engineering from Ecole Supérieure d’Electricité, Paris, France.

Ajay : What would your advice be to young professionals entering the job world today ?

Roger : If you are talking about Statisticians, I would tell them to concentrate on the data and the process rather than on the statistical orthodoxy

Ajay : What interested you most in being the head of KXEN. What is the best feature you like in KXEN. – both as a company and as a product.

Roger :To Make it happen !! Data mining is at its infancy, because SAS and others made it difficult to work with !! they made for an elite of people !!

KXEN role is to open this bottleneck and give power to the users – Analysts will help to train business users and get them confident with their findings.

As a product, I am always suprised by the quality of KXEN results in a fraction of the time compared to first generation workbench and automatically !!

Ajay : What areas has KXEN been most suitable for ? Biggest sucess story so far.

Roger : Classification, regression with thousands of variables and tricky data sets !! We have hundreds of success stories 

Ajay :Could you also comment on how the slowdown and recession would affect the analytics world in terms of newer solutions , Software as a service , more acceptance of trying out the unfamiliar etc ?

Roger : I believe the recession and the slowdown will push analytics further and particularly KXEN approach , since we allow corporations to do much more with less or with the same Team. We are seeing many Analytics Group being reduced and people calling on us to deliver what need to be delivered!!

Ajay : What areas would you rather not recommend KXEN? What other softwares would you recommend in those cases ?

Roger : I would not recommend KXEN in genetics – SVM would be more apropriate

Ajay : Asia has a nascent but high potential market. What are you Asian plans and any clients /case studies here ?

Roger:  We have a presence in every countries but in India – Japan is by far our best country and we have there a fantastic Distributor -WE also have Customers in China , in Asean too – We are looking for a good Distributor in India , but this seems quite difficult.(Note from Ajay – I decided to apply straight away)

Ajay : What is the biggest challenge you have faced while introducing KXEN to a wider audience.
Roger: THe resistance to Change and the fear of classical statisticians that they will loose their job !! in fact this never happened and on  the contrary they become hero in their Corporation after adopting KXEN

  Founder and Chief Executive Officer

Mr. Haddad is responsible for overseeing the KXEN sales team, the distribution channel management, as well as the direction of the company and the strategic growth of the organization. With more than 30 years experience as an industry expert, Mr. Haddad is a forward-looking entrepreneur with an expertise in successfully running companies with multiple channels
of distribution. Mr. Haddad has a long and successful track record in developing new companies into profitable enterprises.

1) What gave you the idea of Life Mojo. 

Well, it is an interesting story, I got a chance to meet Dr. Shikha Sharma (a renowned weight management and health expert) as a Technology Advisor. Dr Sharma wanted a solution for a few problems that existed in her business. After that meeting and after seeing Adnan Sami lose weight, I tried losing weight and was able to lose 11 Kgs in 45 days (which I soon regained).

On analysis of this successful weight loss, the LifeMojo team realized that the key was constant monitoring of what I ate, my fitness regimen, and an eye on my weight. But the problem was that I soon regained weight. The team looked around and found a lot of similar people and a solution to this problem. The solution is right diet consultation, as per one’s body composition and meal preferences. But this right advice is (1) expensive, (2) inconvenient due to the traveling involved (3) inaccessible, since good nutrition experts who understand one’s meal preferences are not easy to find. And thus as a solution to all these problems, LifeMojo was born.

LifeMojo started as a B2B solution for health experts and then got evolved into the present B2C product where people can plan and track their lifestyle and analyze how to have a healthier lifestyle.

How does it work.

On LifeMojo you can start by analyzing your body and lifestyle. You can learn more about your body, daily calorie intake, etc. We have tools that enable you to plan your diet and exercise schedule depending on your analysis, and track them to see the amount of calories you consume and burn. This continuous monitoring of your lifestyle would help you achieve your fitness goals faster.

Do you have international ( non Indian visitors)

Yes, there have been international (non Indian) visitors on LifeMojo (though not a significant number), most of them have discovered LifeMojo by Google searches on topics like running and diet tips.

2) What prompted for you to choose startups rather than work for mainstream IT companies.

I have had a lot of ideas that I wanted to work on, since my third year at NSIT. In order to make them a reality I thought I would have to gain experience on making a product end to end, so I worked for Trilogy (one of the mainstream IT company) for 2 years which was a great learning experience. After that I co-founded a company called MyDuniya and soon realized an idea or a product by itself is not the only thing that one needs to run a company. I decided to work at a startup Flightraja (now known as Via), founded by few of my friends at Trilogy, and experienced how to build and grow a business around a product. I worked there for 1.5 years and learned a lot about what it takes to build a successful company. Around 6 months back I along with other 2 friends Varun (who has been with me since NSIT and we have been working together in the same companies there after) and Himanshu (who has been a great mentor and friend while I was working at Trilogy) started LifeMojo.

So in short we have always wanted to try out our Ideas and a good mix of experiences from mainstream IT company, startup and previous venture is what we have to build and grow LifeMojo.

3) I noticed all founders of life Mojo went to the same college and worked at the same company. How important is this chemistry in founding a startup.

I have seen examples of both for and against "starting a company with friends", but for us having a good chemistry and understanding about each other’s strengths and weaknesses has been very helpful. There are times in a startup when things aren’t as

clear as you want them to be, and it is very crucial to trust each other on every decision to move fast. The chemistry helps us to communicate and take each other’s feedback positively and improve our decisions, which is very crucial for us to grow fast and in the right direction.

4) Whats the toughest point you have had during your career/s and how did you deal with it.

The toughest point in my professional life has been to move out of my first venture MyDuniya. Once I realized that I had to learn a lot before I could build a successful company, it became easier to move on.

5) What would your advice be to young engineers entering college. What would it be for engineers leaving college.

For engineers entering the college, I would recommend "Along with your studies if you could figure out what is it that you want to do with your life, it would be great! Because then it would make a lot of sense for you to concentrate on the stuff that would help you achieve your goals."

For engineers leaving college, I would recommend "Now that you are ready, go do what you aspire to do"

6) What web analytics softwares have you found handy over your career with examples of the sites and a brief description.

For the first 2.5 years at trilogy, I worked on proprietary Trilogy software and the analytics part for it was developed in house as well. During MyDuniya we used Google Analytics and we have been using that till now. We have recently started experimenting with click-heat to see where users are clicking on LifeMojo, we are still in the process of setting it up.

7) I noticed another company by Trilogy alumni called www.lifeblob.com . Any resemblance or co incidence.

No, the similarity in the name is just a co-incidence. In fact, Pranav, Founder and CEO of LifeBlob is also from NSIT (2000 Batch) and was one of my interviewers who recruited me. He is a good friend and was a great mentor for us at Trilogy.

We started off with flab.in (which had a negative connotation), then moved onto healthboon.com (sounded like a hospital) and finally landed up with lifemojo.com. More on it here -> http://blog.lifemojo.com/2008/10/12/whats-in-the-name/

Any thing about venture capital funds plans etc, or any incident you want to share during building up the startup or any customer testimonial.

Customer Testimonial for our B2B offering LifeMojoPro.com:
"iStrait is helping us streamline our operations and better define our promises to our customers with their unique software specially designed for our weight loss and diabetic clients"
- Dr. Shikha Sharma (renowned weight management and health expert)
[Our company is called iStrait Software Solutions Pvt. Ltd.]

LifeMojo helps you achieve a Healthy Lifestyle. LifeMojo provides tools for body analysis to help you know your body. It helps you plan your fitness regimen and monitor your progress. It also provides calorific value of various food items, articles about healthy lifestyle and health tips. And all this is available at no cost!

From the left: Himanshu Khurana, Varun Gupta and Namit Nangia

Namit Nangia, did his BE Computer Science from NSIT Delhi, graduated in 2004 and joined Trilogy. Worked there for 2.5 years and then moved on to start his first venture DigitalStrait (now known as MyDuniya). He tried his luck for 4 months and joined FlightRaja(now known as Via) to learn the ropes of the startup world. Left FlightRaja in May 2008 and started iStrait.

Himanshu Khurana, did his BE Computer Science from NSIT Delhi, graduated in 2003 and joined Trilogy. Worked at Trilogy for 3 years and went on to experience the startup world at ngpay(JiGrahak) . Worked there for 2 years and started iStrait.

Varun Gupta, did his BE Computer Science from NSIT Delhi, graduated in 2004 and joined Trilogy. Worked there for 2.5 years and then moved on to start his first venture DigitalStrait (now known as MyDuniya). He tried his luck for 4 months and joined FlightRaja(now known as Via) to learn the ropes of the startup world. Left FlightRaja in May 2008 and started iStrait.

There are other sides of this story as well.

Second generations of Indian Americans find it hard to adjust to conservative traditional homes and upbringing, while dealing with American modern peer pressure.

In fact there is an acronym for this called ABCD or American Born Confused Desi (Desi is slang in Hindi for Indian Americans among themselves.It literally means local). India is a big outsourcing center and people vent anger on losing jobs to Indian Americans. 

Yet, Indian Americans are equally at risk of their jobs being outsourced. Indian Sikhs were racially attacked after September 11 because they look like Arabs with their turbans, even though ironically Indian Sikhism was created a mechanism against Muslim oppression in pre-British India.

India and the US as foreign policy partners face joint terror threats from jihadis hiding in Pakistan but have differed on approaches to handle this. Given their cultural traditions, Indian Americans have the highest median income levels and lowest per capita crime rates.
The United States has the best educational infrastructure and has the biggest market for goods and services. Indian companies aspiring to grow sales in these markets find it difficult to manage the cultural intricacies and subtle communication for negotiations.

Thus Indian companies have hired an increasing number of American citizens to give that insight among marketing to that market. Similarly countries wanting to enter the Indian market are happy to hire Indian Americans who are aware of working in both cultures. This is also especially true in the outsourcing sector where Indian Americans who have returned are in great demand, because they have the expertise as well as the networking needed.

Salaries in India are lower, top executives can expect unto 100,000-150000 USD only.

But because cost of living in India is so low, you can live in a big bungalow , have a chauffeur driven car,have full time day care for your kids, and a full time butler. The savings that you pile up are also substantial as each dollar is worth 45 Indian rupees.Culturally India has the largest English speaking population, so language is no barrier. They are friendly to foreigners because of the past colonial legacy (The British left in a peaceful transition thanks to Gandhian non violence).

The food and local cuisine are the best, and for a returning expatriate the profile of job and responsibilities is more attractive because of explosive growth and thus faster

opportunities without any stagnation for promotions.

Recently, the US signed a deal to give India nuclear energy components so as to enable it to move to non carbon fuel sources. Indian economy has been booming at 7 % annually for past 6 years, and American companies have used them as good markets for selling produce.  Culturally both countries are similar in terms of democracy: free press and a liking for Movies, Burgers and Music. India has one the largest English speaking population and this helps especially English and American tourists.

Given India’s recent economic rise and aging Indian Americans, many of them have thought of moving back to India. A saying goes you can take the Indian out of India, but cannot take the India out of the Indian. We took a sample of interviews with people who came back and are featuring two interviews

Some people who have seen both sides of the Atlantic and Indian Oceans.

Jagjit Singh (name disguised) emigrated to USA 18 years with his doctor wife, and recently moved back with his two teenage sons to head an India enterprise taking advantage of India’s economic boom. Jagjit is an IVY league MBA graduate with bachelors in engineering. Here is his story

1)      What were the reasons that made you leave India for America . Describe that journey
The tremendous growth opportunity. Due to a variety of factors, the tide is rising in India and a new chapter of the India growth story is being written. In contrast the US is facing stagnation. So, the real action is in India

2)      What were the reasons that made you leave America for India.
You can say the opposite was the case in India 18 years ago

3) Could you give some insights do’s and donts for people who  want to come back, but keep on postponing it.

India is not a bed of roses, things are improving but not as fast as people could imagine. If you can reconcile some basic facts about comforts of living in India then the move will be less mentally straining. The other thing to keep in mind is that the family’s perspective on the move will be very different. You, the professional will see the upsides, they will see the downsides. So preparing the family is also very important  (By doing what?)

4)      How would you rate Indian and American education.

I cannot imagine Indian education being at par with American education, although the gap may be closing. The quality of the Indian students intellect though may be at par with anyone else in the world.

5)      What are the pluses and minuses of being an Indian in America
Plusses are all the material comforts that you are not likely to find as easily anywhere else in the world

Minuses, it is not your culture, the upbringing of your kids for one could be challenging

  (Ajay – This is because of conservative Indian views towards teenage pre-marital sex and insistence to obedience to elders.This is one hidden reason so many Indians become engineers and doctors because they are told by their parents to study for that.)

6)      Should young people in India aspire to go West. Why or Why not ?

Most definitely yes. This to see how certain things ought to be.

(Ajay- Things like non corrupt politicians , a efficient bureaucracy,  even the IRS ,the road infrastructure and the speed at which justice is dispensed in courts.) That will help bring about the necessary change in India faster

Interview 2 

Amit Goel (Name Disguised)  is an IIT (Indian Institute of Technology -there are six of them -Indian equivalents of top engineering schools like MIT and Georgia Tech )graduate and he did his MBA from an American University too. He worked for a decade in the United States including with the best investment banks  , before heading back to India. His wife is also a doctor , and Amit is currently leading an outsourcing team for equity research.

1) What were the reasons that made you leave I

ndia for America . Describe that journey -

At the time I left India, opportunities in India were very limited. The preferred options were IIMs (Indian Institute of Managements -There are seven of them – Indian equivalents of the IVY league business schools) or US universities. Only if one couldn’t pursue either of these two, he or she took up a job in India but would continue to pursue the first two options.

2) What were the reasons that made you leave America for India. - 

My reasons were purely personal. It certainly helped that there are real opportunities that one can pursue these days. Otherwise 10-15 years ago also people wanted to move back but would keep postponing their decision.

3) How would you rate Indian and American education. 

Indian education prepares you well on theoretical or fundamental aspects while American education prepares you much better on the practical aspects. The other difference is that since people pay for education in the US, they take it very seriously especially at the post-graduate level.

 
4) What are the pluses and minuses of being an Indian in America - 

There was a time that being Indian was a definite handicap. Now on the other hand, all Indians are perceived to be intelligent (even if one is just average), so at least in the large metropolitan areas, one feels completely at ease.

5) Should young people in India aspire to go West. Why or Why not ?

People should still go for the right reasons, e.g. higher education, work experience, exposure to different culture, etc. But one should be very thoughtful about whether he or she wants to settle down in the West for good or not.

So these were some perspectives on this fast growing segment of people. We do not claim that this is a generalization for a nation of one billion Indians or one million Indian Americans. But this is a recent trend. Given the fact that a lot of Wall Street Banks like Lehman Brothers employed Indians in substantial numbers, the trend of Indian Americans is likely to increase and so is the number of Indian students going the other way. Perhaps this people to people interaction will help build bridges just as the Rhodes Scholarship helped America and Britain turn from adversaries to friends a century back. Now if only Christopher Columbus had not turned left but right to find India, this story would be different. Had the French defeated Britain , two centuries ago, this story would have been in French. Perhaps these two great nations and people were destined to come closer together.

Additional Section –

Why do well settled young American Indians go back to India ?

Unlike relatively older middle aged Indian Americans, younger Indian Americans who are unmarried go back to India based on following reasons –

Health care of aging parents – Indians have close knit families and children and parents communicate lifelong unlike the West. Health of aging parents is the prime reason for which young students, or Indian Americans go back. Note this is based upon interviews of a few people and anecdotal evidence. Overall parental health, family ties and nostalgia is a big reason for people to leave cushier lifestyles in the United States.

Marriage – Eastern countries like India are conservative in the way women dress and behave, and there is sometimes a disconnect between how they are brought up and the way Indians see women behave their. Unlike other countries, Indians come from democratic countries therefore they acknowledge everyone has the right to behave like this. Besides it can be very boring if all women behave conservatively. However when it comes to marriage, Indian Americans even highly educated ones prefer to around 60-80 % for an arranged marriage in which their parent set up a blind date with someone who is from a similar background. Arranged marriages are not as strict now, there is usually a courtship period of around six to ten months to check for compatibility. An overwhelming percentage of Indian Americans feel uncomfortable with the idea of marrying someone who has slept around with many people before marriage, and they themselves indulge in relatively monogamist behavior.

Economic Crisis and Indian Americans-

The Indian stock market crashed just as the United States market did. The United States dollar however became stronger against the Indian rupee by 20 %, thus making exports from India (mostly software and services cheaper). Indian economy is a bit like United States economy , we export services like movies, software and music and we import a lot of oil (around 80 %). There have been many job cuts in India because of job cuts in America, as back offices generally support people in America. Theoverall feeling is optimistic though, as people belive that the new steps by both the United States federal reserve and India’s reserve bank (which also cut interest rates), make sense, that the worst of this crisis is over, and we have seen worst crisis like 9/11 economic crash and came out of it together.

Summary- These articles represent interviews conducted by me on Americans of Indian descent settled in America, who came back to India or stayed there. I hope it gave you some insight in understanding this fast growing community, which is quite low profile by habit and culture. Remember India and America have a connection going back from the days Columbus took a right turn (he thought he was going to India) and forgot to ask directions. In challenging times like today’s world , these two great people are doing good coming together.

Disclaimer- I traveled to Austin ,Texas, and to  Newark to collect some part of the America based responses, while the India based responses were mostly Bangalore and Delhi based. Thus there is an urban bias in this sample.

Note-Robert Muenchen (pronounced Min’-chen) is the author of the famous R for SAS and SPSS users, and his forthcoming book is an extensive tutorial on anyone wanting to learn either SAS,SPSS,or R or even to migrate from one platform to another. In an exclusive interview Bob agreed to answer some questions on the book , and on students planning to enter science careers.

What made you write the R For SAS and SPSS users?

The book-

A few years ago, all my colleagues seemed to be suddenly talking about R. Had I tried it? What did I think? Wasn’t it amazing? I searched around for a review and found an article by Patrick Burns, “R Relative to Statistics Packages” which is posted on the UCLA site (http://www.ats.ucla.edu/stat/technicalreports/). That article pointed out the many advantages of R and in it Burns claimed that knowing a standard statistics package interfered with learning R. That article really got my interest up. Pat’s article was a rejoinder to “Strategically using General Purpose Statistics Packages: A Look at Stata, SAS and SPSS” by Michael Mitchell, then the manager of statistical consulting at UCLA (it’s at that same site). In it he said little about R, other than he had “enormous difficulties” learning it that he had especially found the documentation lacking.

I dove in and started learning R. It was incredibly hard work, most of which was caused by my expectations of how I thought it ought to work. I did have a lot to “unlearn” but once I figured a certain step out, I could see that explaining it to another SAS or SPSS user would be relatively easy. I started keeping notes on these differences for myself initially. I finally posted them on the Internet as the first version of R for SAS and SPSS Users. It was only 80 pages and much of its explanation was in the form of extensive R program comments. I provided 27 example programs, each done in SAS, SPSS and R. A person could see how they differed, topic by topic. When a person ran the sections of the R programs and read all the comments, he or she would learn how R worked.

A web page counter on that document showed it was getting about 10,000 hits a month. That translates into about 300 users, paging back and forth through the document. An editor from Springer emailed me to ask if I could make it a book. I said it might be 150 pages when I wrote out the prose to replace all the comments. It turned out to be 480 pages!

What are the salient points in this book ?

The main point is that having R taught to you using terms you already know will make R much easier to learn. SAS and SPSS concepts are used in the body of the book as well as the table of contents, the index and even the glossary. For example, the table of contents has an entry for “Value Labels or Formats” even though R uses neither of those terms as SPSS and SAS do, respectively. The index alone took over 80 hours to compile because it is important for people to be able to look up things like “length” as both a SAS statement and as an R function. The glossary defines R terms using SAS/SPSS jargon and then again using proper R definitions.

SAS and SPSS each have five main parts: 1) commands to read and manage data, 2) procedures for statistics & graphics, 3) output management systems that allow you to use output as input to other analyses, 4) a macro language to automate the above steps and finally 5) a matrix language to help you extend the packages. All five of these parts use different statements and rules that do not apply to the others. Due to the complexity of all this, many SAS and SPSS users never get past the first two parts.

R instead has all these functions unified into a common single structure. That makes it much more flexible and powerful. This claim may seem to be a matter of opinion, but the evidence to back it up comes from the companies themselves. The developers at SAS Institute and SPSS Inc. don’t write their procedures in their own languages, R developers do.

How do you think R will impact the statistical software vendors?

With more statistical procedures than any other package, and its free price, some people think R will put many of the proprietary vendors out of business. R is a tsunami coming at the vendors and how they respond will determine their future. Take SPSS Inc. for example. They have written an excellent interface to R that lets you transfer your data back and forth, letting you run R functions in the middle of your SPSS programs. I show how to use it in my book. Starting with SPSS 17, you can also add R functions to the SPSS menus. This is particularly important because most SPSS users prefer to use menus. The company itself is adding menus to R functions, letting them rapidly expand SPSS’ capabilities at very little expense. They saw the R tsunami coming and they hopped on a surfboard to make the most of it. I think this attitude will help them thrive in the future.

SAS Institute so far as been ignoring R. That means if you need to use an analytic method that is only available in R, you must learn much more R than an SPSS user would. Once you have done that, you might be much more likely to switch over completely to R. Colleagues inside SAS Institute tell me they are debating whether they should follow SPSS’ lead and write a link to R. This has already been done by MineQuest, LLC (see http://www.minequest.com/Products.html ) with their amusingly named, “A bridge to R” product (playing off “A Bridge Too Far.”)

Statistica is officially supporting R. You can read about the details at (http://www.statsoft.com/industries/Rlanguage.htm) . Statacorp has not supported R in Stata yet, although a user, Roger Newson, has written an R interface to it (http://ideas.repec.org/c/boc/bocode/s456847.html).

The company with the most to lose are the makers of S-PLUS. That was Insightful Corp. until they were recently bought out by Tibco. Since R is an implementation of the S language, S-PLUS could be hit pretty hard. On the other hand, they do have functions that handle “big data” so there is a chance that people will develop programs in R, run out of memory and then end up porting them to S-PLUS. S-PLUS also has a more comprehensive graphical user interface than R does, giving them an advantage. However, XL-Solutions Corp. has their new R-PLUS version that adds a slick GUI to R (http://www.experience-rplus.com/). There could be a rocky road ahead for S-PLUS. IBM faced a similar dilemma when computing hardware started becoming commodities. They prospered by making up the difference with service income. Perhaps Tibco can too.

Do you have special discounts for students?

My original version of R for SAS and SPSS Users is still online at http://RforSASandSPSSusers.com so students can get it there for free. The book version has a small market that is mostly students so pricing was set with that in mind.

What made you choose a career in Science and what have been the reasons for your success in it.

I started out as an accounting major. I was lucky enough to have had two years of bookkeeping in high school, and I worked part-time in the accounting department of ServiceMaster Industries for several years. I got to fill in for whoever was on vacation, so I got a broad range of accounting experience. I also got my first experience with statistics by helping the auditors. We took a stratified sample of transactions. With transactions divided into segments by their value, and sample a greater proportion as the value increased. For the most expensive transactions, we examined them all. My job was to be the “gofer” who collected all the invoices, checks, etc. to prove that the transactions were real. For a kid in high school, that was great fun!

By the time I was a freshman at Bradley University, I became excited by three new areas: mathematics, computing and psychology. I got to work in a lab at the Peoria Addictions Research Institute, studying addiction in rats and the parts of the brain that were involved. I wrote a simple stat package in FORTRAN to analyze data. After getting my B.A. in psychology, I worked on a PhD in Educational Psychology at Arizona State University. I loved that field and did well, but the job market for professors in that field was horrible at the time. So I transferred to a PhD program in Industrial/Organizational Psychology at The University of Tennessee. It turned out that I did not really care for that area at all, and I spent much of my time studying computing and calculus. My assistantship was with the Department of Statistics. By the time my first year was up, I transferred to statistics. At the time the department lacked a PhD program, so after four years of grad school I stopped with an M.S. in Statistics and got a job as a computing consultant helping people with their SAS, SPSS and STATGRAPHICS programs. Later I was able to expand that role, creating a full-fledged statistical consulting center in partnership with the Department of Statistics. Ongoing funding cuts have been chipping away at that concept though.

What made me a success? I love my job! I get to work with a lot of smart scientists and their grad students, expanding scientific knowledge. What could be better?

Science is boring, and not well paying career compared to being a lawyer or a sales job. People think you are a nerd. Please comment based on your experiences.

Science is constantly making new discoveries. That’s not boring! An area that most people can relate to is medicine. When we finish a study that shows a new treatment is better than an old one, our efforts will help thousands of people. In one study we compared a new, very expensive anti-nausea drug to an old one that was quite cheap. The pharmaceutical company claimed the new drug was better of course, but our study showed that it was not. That ended up helping to control health care costs that we all see escalating rapidly.

Another study found for the first time, a measure that could predict how well a hearing aid would help a person. Now, it’s easy to measure a hearing aid and see that it is doing what it is supposed to do, but a huge proportion of people who buy them don’t like them and stop wearing them after a brief period. Scientists tried for decades to predict which people would not be good candidates for hearing aids. A very sharp scientist at UT, Anna Nabelek, came up with the concept of Acceptable Noise Level. We measured how much background noise people were willing to tolerate before trying a hearing aid. That allowed us to develop a model that could predict well for the first time if someone should bother spending up to $5,000 for hearing aids. For retired people on a fixed income, that was an important finding. An audiology journal devoted an entire issue to the work.

It’s true that you can make more money in many other fields. But the excitement of discovery and the feeling that I’m helping to extend science very satisfying and well worth the lower salary. Plus, having a job in science means you will never have a chance to get bored!

What is your view on Rice University’s initiatives to create open source textbooks at http://cnx.org/ .

I think this is a really good idea. One of my favorite statistics books is Statnotes: Topics in Multivariate Analysis, by G David Garson. You can read it for free at http://www2.chass.ncsu.edu/garson/pa765/statnote.htm .

Universities pay professors to spend their time doing research, which must be published to get credit. So why not pay professors to write text books too? There have been probably hundreds of introductory books in every imaginable field. They cannot all make it in the marketplace so when they drop out of publication, why not make them available for free? I still have my old Introductory Statistics textbook from 30 years ago and the material is still good. It may be missing a few modern things like boxplots, but it would not take much effort to bring it up to date.

I’m also a huge fan of Project Gutenburg (http://www.archive.org/details/gutenberg). That is a collection of over 20,000 books, articles, etc. available there for free download. My wife does volunteer project management and post-processing with Distributed Proofreaders (http://www.pgdp.net/) which supplies books for Gutenburg.

What are your views on students uploading scanned copies of books to torrent sharing web sites because of expensive books.

The cost of textbooks has gotten out of hand. I think students should pressure universities and professors to consider cheaper alternatives. However scanning books putting them up on web sites isn’t sharing, it’s stealing. I put in most of my weekends and nights for 2 ½ years on my book that will be lucky to sell a few thousand copies. That works out to pennies per hour. Seeing it scanned in would be quite depressing.

When is the book coming out ? What is taking so long ?

We ran into problems when the book was translated from Microsoft Word to LaTeX. The translator program did not anticipate that an index would already be in place. That resulted in 2-3 errors per page. We’re working through that and should finally get it printed in early October.

Biography

Robert A. Muenchen is a consulting statistician with 28 years of experience. He is currently the manager of the Statistical Consulting Center at the University of Tennessee. He holds a B.A. in Psychology and an M.S. in Statistics. Bob has conducted research for a variety of public and private organizations and has assisted on more than 1,000 graduate theses and dissertations. He has coauthored over 40 articles published in scientific journals and conference proceedings. Bob has served on the advisory boards of SPSS Inc., the Statistical Graphics Corporation and PC Week Magazine. His suggested improvements have been incorporated into SAS, SPSS, JMP, STATGRAPHICS and several R packages. His research interests include statistical computing, data graphics and visualization,text analysis, data mining, psychometrics and resampling.

Ajay-He is also a very modest and great human being.

http://www.amazon.com/SAS-SPSS-Users-Statistics-Computing/dp/0387094172/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1217456813&sr=8-1

1) What has been the story of your career and what message would you like to send to young people aspiring for MBA’s

Well, a quick history is that I completed my B.Com with specialisation in Accounting & Financial Management from Mumbai University and then my MBA from IIMA in 1993. My first job was with Trans Indus, a UK-based Tour Operator providing tailor-made Independent Holidays to British Travellers. My role was marketing and sales. I then joined Magnasound India to pursue my passion in Music and after 2 years, moved to become part of Sony Music’s set-up team in India. I was soon tempted by the opportunity to explore the emerging Internet space with Mudra Communications, where I was asked to set-up Mudra Interactive. 3 years and a few successful websites and campaigns later, I decided to go my own way and set-up Indigo Consulting in 2000.

My message to young aspiring MBAs is:

1. Please work for at least 2 years before doing an MBA (3-4 years is ideal)

2. After your MBA, forget that you have the degree. Don’t be bound by the "MBA-approved" career options – choose what gives you max satisfaction, cos that is the best guarantee that you will excel at your job.

(PS, you can find more at http://www.rediff.com/money/2007/mar/28guest.htm)

2) What various ventures have you been involved in. What is the motivation/drive behind these initiatives ?

Indigo Consulting (www.indigo.co.in) and Cashcow.in

The motivation behind both these initiatives is to create a commercial venture from scratch that offers sustainable value to clients and employees.

Indigo Consulting: By being the most respected Digital Agency in the Asia-Pacific region – respected through: Innovative Work that delivers business results for client, and provides an experience filled with learning and passion for the employees.

Cashcow.in: By being a definitive resource for Financial Services practitioners that provides timely and relevant information to help them keep pace with the ever changing world of marketing.

Creating this from scratch was personally important to me since my own motivation was the entire creative and "nurturing" process.

Perhaps the biggest risk to what I had started out to do came when 2.5/3 years into the business (and we were stable but yet quite small), my partner and co-founder decided he wanted to pursue other interests instead. I was extremely worried about the impact this would have on the confidence of our clients and employees. We had just about started building a name for ourselves and it would have been easy to have read this as a signal of yet another venture falling by the wayside. Most importantly it challenged my own self-confidence to go it alone and to achieve what we had set out to do. The commitment to the venture and my passion to create the venture is what saw me through those days – and  of course a whole lot of luck which you cannot discount.

3) What are the key learnings that you have learnt in this short period. What advice would you give to someone just out of college today and looking for career choices.

One of the most important learnings is that it is very important to identify and recognise your talents and choose a career that exploits those talents. You have to love what you do to be great at it.

Another lesson that I have learnt is the importance of honesty and integrity towards your work. Your work has to be an expression of who you are, and I’m sure, everyone wants to be known in a positive way for their work.

4) Is an IIM MBA necessary for success .Comments please.Please give supporting anecdotes or examples.

Not at all. There are sufficient examples out there of people who did not have an MBA at all, let alone an IIM MBA – Narayan Murthy is the best example of our times. However, an MBA MAY help in being better prepared, and most importantly as an insurance policy.

5) Whats your vision for Cashcow.in . Which international website is a good comparable for it ? What learnings would one gain from this site or subscribing to the newsletter ?

Cashcow.in aims to be the definitive source for tracking financial services marketing trends and happenings in India, serving as an invaluable resource to practitioners and students of BFSI marketing. It aims to be recognised for its timely, insightful as well as unique coverage of the domain. One of the sites we like in the domain is www.netbanker.com.

Biography- Vikas completed his post-graduation in Business Management from Indian Institute of Management (IIM) Ahmedabad, and has over 14 years’ experience in Sales, Marketing & Interactive Media. Prior to starting Indigo Consulting, he set-up and led Mudra Interactive, Mudra Communications’ Interactive and New Media division.
A music buff, Vikas counts playing guitar, traveling/outdoor activities, and gadgets among his passions. He also runs Cashcow, India’s first blog on Financial Services marketing.

One of World’s Leading and Well Known Authority on Web Analytics

1) What’s the latest trend you see in Online Analytics over the next year and next three to five years.

I strongly believe that web analytics is on its way to becoming business analytics. In the early days we were solely focused on analyzing clickstream data, but in recent years we’ve relaxed our definition of web analytics to include things like voice of customer
and offline outcome data and multivariate testing. More and more I hear people talking about how online customer interaction fits in with the overall goals of the business rather than as an isolated island of activity. In the future I think we’ll see less of a distinction
between traditional business intelligence and what we currently consider to be the separate field of web analytics.

2) Tell us how you came in this field of work, and what factors made
you succeed.

I entered the field of web analytics in 1999. Like many people who got their start at that time, it happened totally by chance. I had applied for a job as a web developer, but the interviewer thought I’d be perfect for another open position – as a web analyst. I took it just to see what it was like. Here it is a decade later and I’m still in web analytics – so I guess you could say it worked out.

Why is web analytics is a natural match for me? Well, I’ve always felt quite comfortable

serving as the "bridge" between technical people and business people. Having a deep understanding of what’s going on with the data is important, but being able to explain what it means to marketers is even more important. I love that challenge.

3) Most challenging and fun project you ever did (anonymous details)

I often look back fondly on a project I did a few years ago – it was really the perfect storm of fun projects. First of all I got to collaborate with a fellow who’s a really accomplished data analyst; he was a joy to work with and our skills were a great match. Second, the project had fairly tight time constraints, so we did very concentrated work for a couple of months and then it was done (as opposed to those un-fun projects that drag on and on with no conclusion). Third, I had unlimited access to full-detail web activity data for a large media site, so I could ask and answer any question, no matter how complex the query.

The project itself involved examining the behavior of first-time visitors to a variety of web properties, to see if we could draw conclusions about what made someone more or less likely to return to the site within a set period of time. We segmented the data in all kinds of ways, ran it through some neat data visualization tools, then drew conclusions and presented our findings to management. As a final deliverable we created a feed that was to be used as the input for a follow-on data mining project. In the end the business was able to gain new insight about how to increase visitor loyalty, and I got a fun project I could talk about in interviews.

4) Advice to fresh college graduates wanting to join online analytics
as a career- Positive Things, Challenges, Skill Requirements.

If you’re new to web analytics there’s a lot you can do to grow your skills even before you land that first job; here are 5 things you can do today:

* Keep up with web analytics blogs. Each blogger has his/her own unique take on the field, and it’s a good way to learn the lingo and the acronyms that we web analysts commonly use.

* Experiment with free tools. Get Google Analytics and test it out, ditto for Google Site Optimizer, ditto for any other tool you can get for free. While interviewers may still look for experience with a particular commercial tool, you will score bonus points for background
research with free tools.

* Go to Web Analytics Wednesday. This is the absolute best way to meet people in your local area who already do web analytics and can help you get your foot in the door.

* Join the Web Analytics Association. You’ll get access to webcasts and research and discounts, plus you can join the global group of volunteers who are actively working to further our profession.

* Finally, show enthusiasm! It’s not enough to want to get into this field just because there are a lot of open jobs – you have to actually demonstrate that you think the work is interesting.

5) Would you like to visit India for work/travel.

Absolutely! I’ve never been to India but it’s definitely on my wish
list. My parents attended a friend’s wedding in Delhi a few years ago
and they came back with so many good stories and pictures. Also, it
would be an honor to meet the Indian web analysts – like you, Ajay –
who I’ve gotten to know by email.

(Note- Ajay feels happy on the last line)

Biography

1) What has been the story of your career and what message would you like to send to young people aspiring for MBA’s

(Masters of Business Administration)

Jaya -I come from a small town in Bihar, where I completed my class 10th in a residential school close-by. Finished my high school from DPS Ranchi and then joined IIT Kanpur for a B. Tech. in Computer Science & Engineering. Joined IIM Lucknow right after that. Since then I worked with a start-up and then with Google India in Product Management roles, before starting full time on Pothi.com

Message to young people aspiring for an MBA: Don’t aspire for an MBA, because everyone else is doing so, or because people want you to. Aspire for being good at what is in your heart. If you an artist at heart, aspire to be a good artist! If MBA it is, then you probably know what you want to get out of it.

2) Why did you turn down a job from Google and start a startup straight after college

Jaya -I had opted out of placement from the campus and joined a start-up (not my own). There was no offer for Google at that time. I did work for Google later as a Product Manager before starting on my own. The reason for joining a start-up after the college were two fold. Firstly some of the job possibilities that looked very lucrative before joining an MBA turned out to be not so lucrative. I really enjoyed reading the psychology and sociology behind consumer behaviour, how they shape choices etc. But a real life job would have entailed ensuring that people drink more Pepsi and eat more chocolates! After my summer internship I knew I wasn’t going to enjoy it. So, I thought going back to technology and take up something where technological as well as management education could be put to use. The start-up was providing the right platform with a product management role.

Secondly, I always wanted to be an entrepreneur myself and hence working in a start-up looked like a good preparation ground. After about two years since graduating, I found myself ready with an idea I was passionate about, a partner with complementary skills and same passion for the idea and some savings to sustain myself for a few months. So, Pothi.com emerged.

3) What are the key learnings that you have learnt in this short period

Jaya -I think one of the major (and useful) realizations for me has been that start-up (success) and institutional funding are not synonymous. Somehow, people always talk to entrepreneurs and discuss start-ups in terms of how good the business is from a Venture Capital (VC) perspective. One has to realize that VCs are geared towards investing in only certain kinds of businesses. That does not mean there is no demand for some of the other kinds of product and services or that a sustainable business can not be made out of them.

4) Is an IIM MBA necessary for success .Comments please.

Jaya-I am very biased against an MBA, especially the way it is used in India/IIMs – as a job agency and not a learning ground. No, I don’t think that an IIM MBA or any MBA (or any particular degree for that matter!) is necessary for success.

Note- IIM stands for Indian Institutes of Management. They are six IIMs and they are the analogous counterparts of the IVY league of USA

5) Whats your vision for Pothi.com . Which website is a good analogy for it ? Why should anyone visit the website ?

Jaya-There is lot more interesting and good content out there than what traditional publishing (and distribution, as we learned over time) can support. All of them should be produced and distributed in the form of a book, which is the most versatile, economical and easy form for consuming content. And Pothi.com should pioneer that!

Jaya Jha is the co-founder of Pothi.com. She is an IIT Kanpur and IIM Lucknow alumnus and was working with Google India as a Product Manager before starting up on Pothi.com full time. She enjoys reading, writing and traveling.She topped IIM Lucknow and was a gold medallist there.First person to be hired outside of US office in the APM Program at Google. APM Pogram is a 2 year training program for fresh graduates to take up the Product Management Role in Google.She was also awarded Sangeeta Goel Memorial Award for highest All India Rank (88) in JEE 2000 amongst the girl students admitted to Indian Institute of Technology, Kanpur

Specific Services at Pothi.com start as low as 12 USD for writers (due to India based costs).International shipping is enabled.

Q1. What has been the story of your career and what message would you like to send to young people aspiring for MBA’s or just starting their careers?

My first initiative, workosaur.com, is a niche job site targeting senior professionals. To exist in the online recruitment industry, which is currently dominated by big players who entered the market early, makes it imperative for me innovate. Moreso because even late entrants with deep pockets have struggled to remain competitive.
2. BUILD TO LAST. DON’T BUILD TO EXIT.
Many Internet startups today are not built on a solid business model. They exist under the assumption that one of the bigger plays will snap them up. When you have such a vision, there is a greater chance that you would fail. Because if things don’t go your way and the bootstrapping costs pile up, you may lose patience, and hang up your boots. So it’s always better to build a business that can go all the way – one that provides tangible value to its audience – so that you can make money at some stage of your product’s lifecycle.
3. WORK HARD.
During the start-up stage, it pays to work hard. For example, I decided to read and learn everything I could – right from managing and hosting a webserver to designing on Photoship/GIMP to programming in PHP/Python to HTMLization to search engine optimization to campaign management and online marketing. I knew enought to be able to do all the initial work myself and was able to launch workosaur.com under USD 1000. If one isn’t willing to do this kind of spadework, it becomes difficult to sustain a start-up which can’t be monetized early.

=
Q4. Is an MBA (IIM or Otherwise) necessary for success. Comments please.

A4. In my opinion, an MBA (especially from an IIM) is detrimental to success if one aspires to be an entrepreneur. Given the prevalence of extremely high salaries amongst the IIM pass-outs, one tends to build a comfort zone around themselves. And it’s difficult to get out of. During conversations I have noticed that most of my peers aspire to start up but the superficial comfort zone of drawing a fixed monthly income holds them back. And once you are so risk-averse, it is very difficult to tread the entrepreneurial route.

Also, the comfort in their current job kills the stomach to do spade work. And with this you have pretty much eliminated the key elements that are necessary to be a successful entrepreneur.
=
Q5. Whats your vision for your site. Which website is a good analogy for it? Why should anyone visit the website?

A5. The vision for workosaur.com is to be the #1 job search destination for middle-level and senior-level management. My current mission is to aggregate the most Rs. 10+ lac jobs and simultaneously build features that would be of great utility for my audience.
It’s a niche segment and currently there is no competitor operating exclusively in the same niche. The competition comes from multiple channels like the top management jobs on naukri and monster plus print classifieds such as Times Ascent.
We have chosen to serve only senior professionals on workosaur. Once you decide to serve a niche, you are in a better position to give them what they want. (We are like a Chinese restaurant which serves good Chinese food to an audience who has come to eat Chinese cuisine.) Senior professionals who visit workosaur will note that we aggregate only 7-figure jobs i.e. jobs that pay more than Rs. 10 lacs per annum. And we do not provide recruiters any access to our database of CVs unless its in response to a job posting. Thus the need for confidentiality is taken care of. Apart from this, you’d find a lot of nifty features that are built for this audience. Log on to workosaur and try them out.

=

Bio:
Workosaur.com is a free job board for senior positions i.e. jobs that pay more than Rs. 10+ lacs per annum. Unlike the other job sites, workosaur.com truly focus on aggregating the most lucrative jobs in India and overseas – Corporate-level, vice president, director and manager positions. Workosaur.com explicitly asks employers to use the job board only for positions for which they can afford to pay more than Rs. 10 lacs per annum, thus making workosaur an aggregator of 7-figure jobs.

What made you enter the online market space ? What prompted you to take up this present assignment?

There is tremendous opportunity in the Indian online market space. For instance buying flight tickets and shopping online was not popular a few years ago. Today people realize that transacting online can be safe, cheap, and can be done within a few seconds. Bankbazaar.com makes getting a great loan bargain as easy as buying an air ticket. Customers using our site will appreciate how much thought we have put into every little detail, which will make the process of applying and getting a loan a hassle free experience. It can help them save precious time and money. We expect a large percentage of consumers from channels such as bank branches and agents to migrate to BankBazaar.com .

Today there are a number of websites that claim to give customers instant loan rate quotes, when they actually just collect customer information and sell them to multiple banks and Direct Sales Agents. In such a scenario, we see a need for BankBazaar.com. Our innovative talent pool drawn from the world’s best colleges and companies are constantly seeking the best possible ways to offer an easier and faster online experience for customers.

I took up this assignment because my team is driving pure innovation in India. We are exploring options that no other financial marketplace in the world has ever attempted before and our goal is to build the world’s premier financial services marketplace (made in India).

What is different about BankBazaar for the customer ? What is different for a prospective employee ?

On BankBazaar.com the customer can instantly get customized competing offers from India’s leading Banks and NBFCs on our secure interface. We are the first neutral provider in the world to have innovated in partnership with leading financial institutions to provide this real-time offer capability. This information is

secure and not shared with any financial institution until the user chooses to apply to a particular financial institution. The users get customised offers, easily compare them and apply for the loan online, all in the span of 15 minutes.

Another critical USP of BankBazaar.com is it’s people. The company was founded by a group of US-returned and homegrown entrepreneurs (http://www.bankbazaar.com/aboutus.html) from Deloitte, Amazon.com, Capital One, Kraft, Microsoft and includes alumni from College of Engineering Guindy, REC-NIT Trichy, Columbia University, Georgia Tech, Virginia Tech and BITS Pilani. This team built the world’s largest online credit card acquisition channel for Amazon.com in the US, and now we are back in India to weed out inefficiencies, streamline and release value in financial delivery channels in India. All employees work in an environment of high innovation, high intelligence and quick action. We offer excellent remuneration packages and an attractive ESOP program in one of India’s fastest growing start-ups.

What are your views on predatory lending and adverse selection ?

Predatory lending is a bad practice that needs to be weeded out from the system. Loan-seekers need to understand all the terms of the loan, which includes the interest rate, other parameters like charges levied under different circumstances, pre-payment terms etc. Another significant factor that needs to be observed is the customer service of the financial institution that offers the loan.

BankBazaar.com provides consumers a very transparent picture displaying all hidden fees and detailing the total cost of loans, which helps the user effectively choose a loan of his preference. We are very confident that loan seekers will love the user friendly and effective interface of our site. The key determinant of how soon BankBazaar.com will take off is based on how quickly we can increase consumer awareness about our product.

What has been your biggest challenge in your career so far ?

Before BankBazaar.com I managed Deloitte Touche & Tomahatsu’s alliance with the world’s leading Information Management Company on the US East Coast. Running such a large corporate alliance was challenging because the sales teams allowed quarterly revenue goals to drive long term sales strategy which was resulting in lost opportunity. My biggest challenge was to guide the sales organizations on both sides of the alliance to invest their time in creating and implementing the alliance sales approach, which sometimes resulted in small revenue reduction in the short term but yielded long term sustainable incremental profits.

What advice would you give to a freshman out of college entering in the corporate world today ?

Try and excel at whatever you do – there is a lot of opportunity for someone who is smart, committed and has the confidence and ability to take the entrepreneurial risk.

Do you think the present economic slowdown will taper down fast ? How long do you think the Indian regulators will take to adopt a higher rate strategy?

Several analysts and Finance Ministry officials have indicated that they expect interest rates to ease up in 2009 – lets wait and watch.

A short biography

Adhil Shetty is the Founder & CEO of BankBazaar.com. Prior to BankBazaar.com, Adhil lived in New York City and managed Deloitte Touche Tomahatsu’s US East alliances with the world’s leading Information Management Company. Earlier in his career, Adhil worked with Cisco Systems as an engineer in Bangalore and San Jose. Adhil has a Masters degree in International Relations with a specialisation in International Finance and Business from Columbia University in the City of New York, and a Bachelors degree in Engineering from the College of Engineering Guindy, Anna University.

Adhil Shetty is CEO & Founder of BankBazaar.com – An online marketplace where you can instantly get loan rate quotes for your personal loan and home loan from India’s leading banks and NBFCs .

1) What prompted you take a career in science, and what has been the reason you stuck to it, and been a success in it ?

I was doing mathematics for fun at a very young age when my friends were interested in sports, cars and movies. When I finished my master, I was approached by one of the professors to pursue a PhD program. It was in statistics (image analysis, bayesian clustering), and I thought that choosing statistics rather than number theory  or numerical analysis would increase my chances of getting a job after presenting my thesis. At that time, my favorite subject was indeed number theory – I was even published in J. of Number Theory. After earning my PhD, I moved to Cambridge, then North Carolina, then the Internet industry – with a very interesting detour into finance and risk management / fraud detection between 2002 and 2005.

thanks to advertising. Membership will continue to be free. I will probably add a data mining search engine or a data mining encyclopedia, as these features help grow organic traffic tremendously.

stakeholders. Luckily for me, I did well in my current assignment, being business head for Simplymarry.com, a C2C matchmaking engine in India.

What advice would you give to some one fresh from college who is trying to take to make a career in internet marketing

Have passion for internet, the growth, the challenge and the money is here for you. The best of all, you do not need any great formal qualification, just the passion and a decent IQ will pull you through.

What challenges and opportunities do you see in Internet in India in next one year ? and in next five years?

The internet connections(and the speed), is the biggest challenge, with only 40 million connections, we are still behind. The growth rate is close to 35%, which is good but not very exciting. I think PC penetration is another issue, which needs to be addressed. The key here lies with government, broadband internet providers and PC manufacturers. But unfortunately things are slow, as compare to what we saw in China. I think in long term the challenge is to leverage internet connectivity into strong business models. The e-commerce, online stores, local fulfillment(click + brick) models need to be in vogue. Another thing is convergence of hardware in terms of mobile and PC, and server based delivery. Just like Japan, we can be the next big nation on GPRS and other ways of connectivity on mobile.

Whats the most admired company in your opinion in Internet in India and in the World and what reasons do you think makes it so..

The company I admire the most from India in internet space is Rediff.com. They have kept the business honest and competitive, I appreciate their courage on coming our with features purely with Indian audience in mind. They have done their basics right and somehow come out as good service providers for the Indian consumers.

Biography

Sandeep Amar has an overall corporate experience of over 11 years in Online media, Print media, broadcasting and Financial domain in India. Sandeep is an MBA from FMS, Delhi, and graduate from Hansraj College, Delhi University,India. He is also a black belt in six sigma. Sandeep is co-author of a book on quantitative techniques by the name "Crack the CAT". Sandeep Amar leads the matchmaking portal simplymarry.com, a Times Group company in India. HPrior to this he worked with Citigroup in India as Assistant Vice President – E-Business, here he was taking care of Internet Marketing, Lead Management Systems, Lead fulfillment and the portal of Citifinancial in India.He also served as Asst. Vice President – Marketing in ZEE Telefilms and as Manager – Brand and Business, for economictimes.com.

An interview with Edith Ohri ,a pioneer in Data Mining from Israel, one of the technologically and educationally advanced countries in Asia.

1) Tell us how you came in this field of work, and what factors made you succeed.
My introduction to data mining was at work right after my MS degree. The CEO of the company asked the IE department to organize the many small customer orders in groups that "make sense" for production, in a remote plant which the company had just opened. At the time, there was no solution available for this request. Whoever got the job was destined to months of laborious work. As a new comer I was assigned that unpopular job. Being new at that place, my problem with it was even greater: I had no knowledge about the production preferences and practices and could not had guessed what to assume and where to focus. This lack of expertise and inability to rely on expert knowledge or past conventions was as things turned out the key for success, as it forced me to rely on objective analysis alone.

2) Most challenging and fun project you ever did (anonymous details).
Every project is fun for me. It is always challenging to crack new complex data especially if previous analysis attempts have little results. In almost every project there is a surprising and interesting twist. I find it every time again very rewarding to discover hidden facts and then rationalize them with the client. Currently I’m working on an extremely large data set that requires farther software development. For me is

like being in a suspense movie, watching it unfolding and gradually falling into place(s). Maybe this last one is the most challenging and fun and… fresh (impressions fade in time)

3) What’s the latest trend you see in data mining over the next year and next three to five years?
My guess about the evolution of data mining is that (a) it will have yet another name after exhausting the term BI; (b) the emphasis will shift to stream data interpretation; (c) Software-as-a-Service will become the preferred choice rather than built in inflexible solutions; and (d) there will be new ventures (hopefully my own) in the area of old data "dump" recycling, i.e. putting old data to use.

4) Advice to fresh college graduates wanting to join data mining as a career.

Data mining is a very difficult field as its theory to my humble opinion is still far from being established. Therefore my advice for students is to be inquisitive, never take a solution model for granted, do not compromise, and remember – it is important particularly in this field – to always see the big picture beyond your client’s requests.

5) Would you like to visit India for work/travel?

A friend of mine had a beautiful vacation in India. I hope to have the opportunity to travel there too.

Biography
Edith Ohri heads a pioneering data-mining company which is dedicated to the application of GT – a new DM solution for unsupervised and complex data. Her background is Industrial & Management Engineering, MSc. She had started researching the issue of data mining in the early 80’s, and has continued with it ever since. She created a new model (GT) which enables larger and more complex data analysis. In 2002 she started in SMU Singapore the development of GT software. She is involved in several areas of implementation, such as: BI, Quality Control, Bio-med and Research. She manages a DM forums with Israel Engineering Association and a DM forum with the Data Warehouse site (Israel). She is a member and active participant in a number of DM forums, give presentations, and write articles.

Note- Edith Ohri is a good friend and pen pal but is not related to Ajay Ohri (www.decisionstats.com). Correlation is not causation.

Ajay- What’s the latest trend you see in Computer Programming over the next year and next three to five years.

Alan- Silverlight and Flex will be huge and will really enable much more SaaS. The current web simply needs wholesale replacement to make it more usable for business applications. These new RIAs will allow us, as developers, to take it to a whole new level. Expect a massive influx of dollars into web redesign and redevelopment.

Ajay-  Tell us how you came in this field of work, and what factors made you succeed.

Alan- I got into computers in high school (this was very early computing). I loved the sense of challenge that computers offered: they were a big crossword puzzle. I succeeded because I never viewed a problem the way a typical computer person or scientist would view them. As a history guy, I took a more holistic approach to problems. Heck, if you don’t know about a particular theory, you won’t be constrained by it. If you do know it, sometimes ignore it to get the job done, even if it isn’t as pretty.

Ajay-  Most challenging and fun project you ever did (anonymous details)

Alan- I have had many, many rewarding projects. As a consultant, every job is different. However, the spare time project one I am currently working on (figuring out the layout of the sas dataset) is perhaps my favorite due to the complexity.

Ajay- Advice to people wanting to join computer programming as a career- Positive Things, Challenges, Skill Requirements.

Alan- First of all, programming is hard so be prepared to work to be good. Never ever stop evolving and looking for the next thing: you are only as good as your last 18 months of experience.

The career is very rewarding since you are continuously facing challenges that must be overcome. Computers have no patience for mistakes so they require a lot of patience for programmers.

Always, always, always think outside of the box. Approach problems differently. If you hit an obstacle, move around it rather than always trying to burrow through. At the end of the day, it is all about getting the job solved at the speed of business not finding a cool, nifty new algorithm: do that on your spare time.

Ajay- Would you like to visit India for work/travel.

Alan- I honestly don’t like to travel long distances. After a long corporate career flying over a million miles, travel is simply taxing to me and takes me away from what I love to do: programming. As a history major, I love various cultures and would enjoy the beauty and history that India provides but would dread the flight ;-]

Bio;

Alan Churchill has been coding in SAS for over 20 years and worked at SAS as a senior consultant for 5 1/2 years.At SAS, Alan worked on the Microsoft-SAS Alliance and helped SAS customers integrate with .NET. He is also responsible for coding the engine for SAS’s web analytics product. Currently, he is the owner of Savian which specializes in Microsoft-SAS solutions. He lives and works in Colorado Springs, Colorado.

Note from Ajay-

Stan Relihan is the 40th most connected person list on LinkedIn out of 25 million people.His podcast: http://connections.thepodcastnetwork.com gets more than 12,000 downloads a month and he has leveraged social networking to maximum advantage for building great assets as well as increasing his own business value. Here he shares some insights  on using social networking networks.

1) What’s the latest trend you see in community sites over the next year and next three to five years.

I see more and more emphasis on the need to focus on Business outcomes -not just idle chit-chat or frivolous questions. Social Networking sites will also need to truly embrace the Web 2.0 ethos – where what the users want is ultimately more important that what the site owner / operators think it should be.  This means more responsiveness to requests for new features and more transparency & intercommunication from management with their users. In the end, just like with Search Engines & Operating Systems, many will cease to exist – and only a few dominant players will continue.

2) Do you think the Ning.com network would be good for all communities?

Biography-

Stan Relihan is President & CEO of Expert Executive Search, one of Asia-Pacific’s best-connected & most influential Executive Search, Selection & Recruitment firms, with an unparalleled network of contacts and business alliances around the world.For organizations that truly believe that people are their greatest assets, Expert Executive Search can deliver key individuals that enable you to outperform your competition.

LINKS: