Any art student knows: learning Video and HTML really helps.
Here is a software called Skitch that helps with Screen Capture.
Online
2009
12.26
12.26
Online Rejection By a Random Tweep
It’s a glass jungle out there in Cyberia
We are all ogling peeping creeps craving sleep
One more I have been proved a fool
By the Adobe touched up eyes and smile
Everything that is great online hardly ever is -wasnt that a rule.
so we return to our PCs Macs and machines
gorging bits and bytes of digital stimulation
We may be aging sugar dandies in reality, but online we will be forever eighteen.
Music for Nothing and your Laptop Software for Free
2009
12.17
12.17
If you love your Apple iPhone / iPod, the latest release of Ubuntu from Canonical called Karmic Koala ( the counter part to Microsoft’s Windows 7) is totally compatible for USB connection.
Note the Ubuntu laptop is cheaper than your Windows or Apple Laptop as the software is free. People say oh it is Linux and so tough but that was a decade back – you can check it out from http://ubuntu.com yourself.
Ubuntu uses Rhythmbox for playing songs and uses FSpot and Gimp for images. The audio quality is excellent ( even if you are using very old hardware like I do) and Gimp is the best image manipulation software -comparable to ones from Adobe- intuitive and easy. You can also use Rhythmbox for listening to internet radio- legit music for free and can use gimp for making fun photos for Facebook
Holiday Season Offers
2009
12.16
12.16
Here is a great offer from the Predictive Analytics Conference. Basically a 15 % discount to visit PAWS in San Fransisco AND a chance to win a free pass as well.
Since I basically need just one pass for the conference- if I cross more than 1 free pass, I will simply give it back to the people who registered using my blog. Simple. However if this had been NFL tickets the offer may not be repeated ( note to myself- I need to get into predictive analytics for sports using the data mining and analytics softwares I have been reading or find a company that has a predictive analytics for SPORTS vertical).
Books of 2009
2009
12.16
12.16
Here are some books that I read, tried to read and should get more time to read.
Non coding Favorite Book ” The Age of Unthinkable”
Coding Favorite Book “R for SAS and SPSS Users” and “Elementary Statistics using JMP”
Calculus is as boring as reading a comic for Christmas
2009
12.11
12.11
If you though understanding Calculus was OMG so Freaking Tough-
here is an Artistic Way to read and improve your Maths.
Why Improve Maths.
Helps you while watching football and baseball with your BFF.
Helps you choose a very good looking mobile with an inexpensive mobile plan.
Helps you calculate the mortgage and credit card rates while reading the fine print.
Helps you get the Insurance or Health Insurance with
lowest premium over highest approval
Nopes- Maths is easy and inexpensive. You can read this for just 75 dollars. All six comics. Or gift it to someone you love who is having trouble with maths.
Screenshot Citation-
Manga Guides ( courtesy Japanese Civilization)
No Starch Press – helped by O Reilley Media Since 2004.
Sarah Blow- Founder of Girlly Geek Dom (http://girlygeekdom.com/ )
Ajay Ohri- Founder of Dude of Data ( http://dudeofdata.com/ )
Art and Maths together: Fractals
2009
11.29
11.29
Thats right. You can use simple mathematics to create beautiful images and art work. There are enough examples in nature for doing so. It is called Fractal.
Basically a Fractal is an image in which some smaller image is repeatedly fitted to create patterns.
See this Triangle Fractal. Basically you are creating patterns by making triangles fit within the triangle.
Now you can use the same technique for a different triangle to create a snowflake. Maths to make drawings of snowflakes – well here you go-
To create a Koch snowflake, one begins with an equilateral triangle and then replaces the middle third of every line segment with a pair of line segments that form an equilateral “bump.” One then performs the same replacement on every line segment of the resulting shape, ad infinitum. With every iteration, the perimeter of this shape increases by one third of the previous length. The Koch snowflake is the result of an infinite number of these iterations, and has an infinite length, while its area remains finite. For this reason, the Koch snowflake and similar constructions were sometimes called “monster curves.” Citation-Wikipedia
Now to create a leaf- see the leaf consists of smaller leafs and smaller leafs.
Hey even nature uses maths here-this is a fractal shape.
is this art- well Jackson Pollock thought so
While Pollock’s paintings appear to be composed of chaotic dripping and splattering, computer analysis has found fractal patterns in his work.[7]
From wikipedia
Here is the equation-
A fractal is “a rough or fragmented geometric shape that can be split into parts, each of which is (at least approximately) a reduced-size copy of the whole,”[1] a property called self-similarity. Roots of mathematical interest in fractals can be traced back to the late 19th Century; however, the term “fractal” was coined by Benoît Mandelbrot in 1975 and was derived from the Latin fractus meaning “broken” or “fractured.” A mathematical fractal is based on an equation that undergoes iteration, a form of feedback based on recursion.[2]
A fractal often has the following features:[3]
- It has a fine structure at arbitrarily small scales.
- It is too irregular to be easily described in traditional Euclidean geometric language.
- It is self-similar (at least approximately or stochastically).
- It has a Hausdorff dimension which is greater than its topological dimension (although this requirement is not met by space-filling curves such as the Hilbert curve).[4]
- It has a simple and recursive definition.
Interviews with Top Analytics Persons
2009
11.27
11.27
Here is a photo of me at the MET. That’s a big big museum in New York
Here is a Facebook Application I am creating for aggregating Blog Feeds ( Note you can do this in less than 2 hours for a full app- I took 15 minutes including uploading and downloading stuff. That’s how easy it is). It is also a valuable skill if you like to get a job for just spending time on Facebook.
http://apps.facebook.com/decisionstats/
And here’s a list of all the interviews I have done-
FUN MATHS
2009
11.26
11.26
Some nice ways to teach kids maths
Random
2009
11.23
11.23
Random
Random poetry and random lines
Cynically wasting non ample time
Been here before and will do it again
Anonymous scribbling to dribble away pain
Cynically wasting non ample time
Been here before and will do it again
Anonymous scribbling to dribble away pain
Loneliness, estrangement and non essential angst
Fulfilled life yet insecure pangs
An analytical brain that runs past the clock
Slippery tongue not prone to small talk
Fulfilled life yet insecure pangs
An analytical brain that runs past the clock
Slippery tongue not prone to small talk
Self analysis and self delusion have an intersecting arc
Common area where insecurities are parked
Common area where insecurities are parked
In the randomness of the moment till we get busy again
Trivial habits to keep us sane.
Trivial habits to keep us sane.
Making gain is more important than fun
I gotta finish this by noon so forgive the pun
Random fun in organized trail paths
Lends spontaneity however temporary it lasts.
I gotta finish this by noon so forgive the pun
Random fun in organized trail paths
Lends spontaneity however temporary it lasts.
The Daily Beacon: University of Tennessee’s student paper
2009
11.10
11.10
And how I became a front page celebrity on the Daily Beacon
The Homework of Dr T
2009
11.09
11.09
The Homework of Dr T
Every Monday and Thursday we meet him
In a room and time pre-ordained
And though we are glad to learn statistics from him
The homework can be such a pain
Dt T and his homework
are like the tasks of Hercules
No sooner have we finished one
That we find the next one has been released
And yes we are young and
Sure this is all good for us
Bit less theory and maybe more application
Could maybe help us find a job for the same
The jobs are all in web analytics, and data mining per se
But the theory is based on 19 th century industry
Old Faithful erupts a dozen times
In a dozen textbooks with faithful regularity
We are a university we give out degrees
Comes the stern paternal decree
In order to convene your thesis committee
Your grades must rise above B
And when I ask
Why dont we try something new
A Hapless shrug comes
over you
You shake your head at my impudence
Then forgive me my young stupidity
Mentally smile and think -just a kid
he needs time to settle his energy
Hard work ,
he should not shirk
Let’s train him hard,
Give him more home-work
( Any resemblance to reality is coincidental.
This is a work of fiction)
Hello Dude!
2009
10.20
10.20
Welcome to Dude of Data.
This site is meant to make analytics fun. For Students, and people who always wanted to learn science but met teachers who made it too tough.Some people say that making science fun is as impossible as landing a woman on the moon.
We’ll see about that!
The smartest guy in high school is not the Quarter Back- it is the nerd who is fiddling with the computers and maths homework.
The smartest parents try and make sure that
maths is as important as dating advice.
Making science fun again.
Interview Michael Zeller,CEO Zementis on PMML
2009
10.05
10.05
Here is a topic specific interview with Micheal Zeller of Zementis on PMML, the de facto standard for data mining.
Ajay- What is PMML?
Mike- The Predictive Model Markup Language (PMML) is the leading standard for statistical and data mining models and supported by all leading analytics vendors and organizations. With PMML, it is straightforward to develop a model on one system using one application and deploy the model on another system using another application. PMML reduces complexity and bridges the gap between development and production deployment of predictive analytics.
PMML is governed by the Data Mining Group (DMG), an independent, vendor led consortium that develops data mining standards
Ajay- Why can PMML help any business?
Mike- PMML ensures business agility with respect to data mining, predictive analytics, and enterprise decision management. It provides one standard, one deployment process, across all applications, projects and business divisions. In this way, business stakeholders, analytic scientists, and IT are finally speaking the same language.
In the current global economic crisis more than ever, a company must become more efficient and optimize business processes to remain competitive. Predictive analytics is widely regarded as the next logical step, implementing more intelligent, real-time decisions across the enterprise.
However, the deployment of decisions based on predictive models and statistical algorithms has been a hurdle for many companies. Typically, it has been a complex, costly process to get such models integrated into operational systems. With the PMML standard, this no longer is the case. PMML simply eliminates the deployment complexity for predictive models.
A standard also provides choices among vendors, allowing us to implement best-of-breed solutions, and creating a common knowledge framework for internal teams – analytics, IT, and business – as well external vendors and consultants. In general, having a solid standard is a sign of a mature analytics industry, creating more options for users and, most importantly, propelling the total analytics market to the next level.
Ajay- Can PMML help your existing software in analytics and BI?
Mike- PMML has been widely accepted among vendors, almost all major analytics and business intelligence vendors already support the standard. If you have any such software package in-house, you most likely have PMML at your disposal already.
For example, you can develop your models in any of the tools that support PMML, e.g., SPSS, SAS, Microstrategy, or IBM, and then deploy that model in ADAPA, which is the Zementis decision engine. Or you can even choose from various open source tools, like R and KNIME.
Ajay- How does Zementis and ADAPA and PMML fit?
Mike- Zementis has been a avid supporter of the PMML standard and is very active in the development of the standard. We contributed to the PMML package for the open source R Project. Furthermore, we created a free PMML Converter tool which helps users to validate and correct PMML files from various vendors and convert legacy PMML files to the latest version of the standard.
Most prominently with ADAPA, Zementis launched the first cloud-computing scoring engine on the Amazon EC2 cloud. ADAPA is a highly scalable deployment, integration and execution platform for PMML-based predictive models. Not only does it give you all the benefits of being fully standards-based, using PMML and web services, but it also leverages the cloud for scalability and cost-effectiveness.
By being a Software as a Service (SaaS) application on Amazon EC2, ADAPA provides extreme flexibility, from casual usage which only costs a few dollars a month all the way to high-volume mission critical enterprise decision management which users can seamlessly launch in the United States or in European data centers.
Ajay- What are some examples where PMML helped companies save money?
Mike- For any consulting company focused on developing predictive analytics models for clients, PMML provides tremendous benefits, both for clients and service provider. In standardizing on PMML, it defines a clear deliverable – a PMML model – which clients can deploy instantly. No fixed requirements on which specific tools to choose for development or deployment, it is only important that the model adheres to the PMML standard which becomes the common interface between the business partners. This eliminates miscommunication and lowers the overall project cost. Another example is where a company has taken advantage of the capability to move models instantly from development to operational deployment. It allows them to quickly update models based on market conditions, say in the area of risk management and fraud detection, or to roll out new marketing campaigns.
Personally, I think the biggest opportunities are still ahead of us as more and more businesses embrace operational predictive analytics. The true value of PMML is to facilitate a real-time decision environment where we leverage predictive models in every business process, at every customer touch point and on-demand to maximize value
Ajay- Where can I find more information about PMML?
Mike- First there is the Data Mining Group (DMG) web site at http://www.dmg.org
I strongly encourage any company that has a significant interest in predictive analytics to become a member and help drive the development of the standard.
We also created a knowledge base of PMML-related information at http://www.predictive-analytics.info and there is a PMML interest group on Linked
In http://www.linkedin.com/groupRegistration?gid=2328634
This group is more geared toward a general discussion forum for business benefits and end-user questions, and it is a great way to get started with PMML.
Last but not least, the Zementis web site at http://www.zementis.com
It contains various PMML example files, the PMML Converter tool, as well links to PMML resource pages on the web.
For more on Michael Zeller and Zementis read his earlier interview at http://decisionstats.wordpress.com/2009/02/03/interview-michael-zeller-ceozementis-2/
Interview Ken O Connor Business Intelligence Consultant
2009
10.05
10.05
Here is an interview with an industry veteran of Business Intelligence, Ken O Connor.
Ajay- Describe your career journey across the full development cycle of Business Intelligence.
Ken- I started my career in the early 80’s in the airline industry, where I worked as an application programmer and later as a systems programmer. I took a computer science degree by night. The airline industry was one of the first to implement computer systems in the ‘60s, and the legacy of being an early adaptor was that airline reservation systems were developed in Assembler. Remarkable as it sounds now, as application programmers, we wrote our own file access methods. Even more remarkable, as systems programmers, we modified the IBM supplied Operating System, originally known as the Airline Control Program (ACP), later renamed as Transaction Processing Facility (TPF). The late ‘80s saw the development of Global “Computer Reservations Systems” (CRS systems) including AMADEUS and GALILEO. I moved from Aer Lingus, a small Irish airline, to work in London on the British Airways systems, to enable the British Airways systems share information and communicate with the new Global CRS systems.
I learnt very important lessons during those years.
* The criticality of standards
* The drive for interoperability of systems
* The drive towards information sharing
* The drive away from bespoke development
In the 90’s I returned to Dublin, where I worked as an independent consultant with IBM on many data intensive projects. On one project I was lead developer in the IBM Dublin Laboratory on the development of the Data Replication tool called “Data Propagator NonRelational”. This tool automatically propagates updates made on IMS databases to DB2 databases. On this project, we successfully piloted using the Cleanroom Development Method, as part of IBM’s derive towards Six Sigma quality.
In the past 15 years I have moved away from IT towards the business. I describe myself as a Hybrid. I believe there is a serious communications gap between business users and IT, and this is a frequent cause of project failures. I seek to bridge that gap. I ensure that requirements are clear, measurable, testable, and capable of being easily understood and signed off by business owners.
One of my favorite programmes was Euro Changeover, This was a hugely data intensive programme. It was the largest changeover undertaken by European Financial Institutions. I worked as an independent consultant with the IBM Euro Centre of Competence. I developed changeover strategies for a number of Irish Enterprises, and was the End to End IT changeover process owner in a major Irish bank. Every application and every data store holding currency sensitive data (not just amounts, but currency signs etc.) had to be converted at exactly the same time to ensure that all systems successfully switched to euro processing on 1st January 2002.
I learnt many, many lasting lessons about data the hard way on Euro Changeover programmes, such as:
* The extent to which seemingly separate applications share operational data – often without the knowledge of the owning application.
* The extent to which business users use (abuse) data fields to hold information never intended for the data field.
* The critical distinction between the underlying data (in a data store) and the information displayed to a business user.
I have worked primarily on what I call “End of food chain” projects and programmes, such as Single View of Customer, data migrations, and data population of repositories for BASEL II and Anti Money Laundering (AML) systems. Business Intelligence is another example of an “End of food chain” project. “End of food-chain” projects share the following characteristics:
* Dependent on existing data
* No control over the quality of existing data they depend on
* No control over the data entry processes by which the data they require is captured.
* The data required may have been captured many years previously.
Recently, I have shared my experience of “Enterprise wide data issues” in a series of posts on my blog, together with a process for assessing the status of those issues within an Enterprise (more details). In my experience, the success of a Business Intelligence programme and the ease with which an Enterprise completes “End of food chain” data dependent programmes directly depends on the status of the common Enterprise Wide data issues I have identified.
Ajay -Describe the educational scene for science graduates in Ireland. What steps do you think governments and universities can do to better teach science and keep young people excited about it?
Ken- I am not in a position to comment on the educational scene for science graduates in Ireland. However, I can say that currently there are insufficient numbers of school children studying science in primary and 2nd level education. There is a need to excite young people about science. There is a need for more interactive science museums, like W5 in Belfast which is hugely successful. Kids love to get involved, and practical science can be great fun.
Ajay- What are some of the key trends in business intelligence that you have seen-
Ken- Since the earliest days of my career, I have seen an ever increasing move towards standards based interoperability of systems, and interchange of data. This has accelerated dramatically in recent years. This is the good news. Further good news is the drive towards the use of external reference databases to verify the accuracy of data, at point of data entry (See blog post on Upstream prevention by Henrik Liliendahl Sørensen). One example of this drive is cloud based verification services from new companies like Ireland based Clavis Technology.
The harsh reality is that “Old hardware goes into museums, while old software goes into production every night”. Enterprises have invested vast amounts of money in legacy applications over decades. These legacy systems access legacy data in legacy data stores. This legacy data will continue to pose challenges in the delivery of Business Intelligence to the Business community that needs it. These challenges will continue to provide opportunities for Data Quality professionals.
Ajay- What is going to be the next fundamental change in this industry in your opinion?
Ken- The financial crisis will result in increased regulatory requirements. This will be good news for the Business Intelligence / Data Quality industry. In time, it will no longer be sufficient to provide the regulator with ‘just’ the information requested. The regulator will want to see the process by which the information was gathered; the process controls, and evidence of the quality the underlying data from which the information was derived. This move will result in funding for Data Governance programmes, which will lead to increased innovation in our industry.
Ajay- Describe your startup Map My Business, your target customer and your vision for it.
Ken- I started MapMyBusiness.com as a “recession buster”. Ireland was hit particularly hard by the financial crisis. I had become over dependent on the financial services industry, and a blanket ban on the use of external consultants left me with no option but to reinvent myself. MapMyBusiness.com helps small businesses to attract clients, by getting them on Google page one. Having been burnt by an over dependence on one industry, my vision is to diversify. I believe that Data Governance is industry independent, and I am focussing on increasing my customer base for my Data Governance consultancy skills, via my company Professional IT Personnel Ltd.
Ajay- What do you do when not working with customers or blogging on your website?
Ken- I try to achieve a reasonable work/life balance. I am married with two children aged 12 and 10, and like to spend time with them, especially outdoors, walking, hiking, playing tennis etc. I am involved in my community, lobbying for improved cycling infrastructure in our area (more details). Ireland, like most countries, is facing an obesity epidemic, due to an increasingly sedentary lifestyle. Too many people get little or no exercise, and don’t have the time, willpower, or perhaps money, to regularly work out in a gym. By including “Active Travel” in our daily lives – by walking or cycling to schools and local amenities, we can get enough physical exercise to prevent obesity, and obesity related health problems. We need to make our cities, towns and villages more pedestrian and cyclist friendly, to encourage “active travel”. My voluntary work in this area introduced me to mapping (see example), and enabled me to set up MapMyBusiness.com.
Biography-
Ken O’Connor is an independent IT Consultant with almost 30 years of work experience. He specialises in Data: Data Migration, Data Population, Data Governance, Data Quality, Data Profiling…His company is called Professional IT Personnel Ltd.
Ken started his blog (Ken O’ Connor Data Consultant) to share his experience and to learn from the experience of others. Dylan Jones, editor of dataqualitypro, describe Ken as a “grizzled veteran”, with almost 30 years experience across the full development lifecycle.
