Microsoft announced earlier this week that a CEP/stream processing product will be included in SQL 2008 R2. Complex Event Processing, or CEP, is primarily an event processing concept that deals with the task of processing multiple events with the goal of identifying the meaningful events within the event cloud. CEP employs techniques such as detection of complex patterns of many events, event correlation and abstraction, event hierarchies, and relationships between events such as causality, membership, and timing, and event-driven processes.
Microsoft called out four reasons to me why CEP might be needed in addition to ordinary database processing. Two are the standard reasons for data reduction:
1. Without CEP, you can’t bang the data into the database fast enough.
2. You don’t want to keep most of the data past a short time window anyway.
The other two are also fairly standard reasons for using CEP:
3. Standard SQL isn’t all that great for time series anyway.
4. CEP use cases often call for incremental processing and/or parameterization of queries, something CEP engines are commonly better designed for than are DBMS.
However, Microsoft seems to be taking a somewhat different approach to time-based SQL extensions than some other vendors. To quote email Microsoft sent today:
Microsoft Research (MSR) introduced the temporal extensions to relational algebra based upon a notion of application time that is independent of system time. It matters when the event originated instead of when they arrived at the processing system. Further it treats each event as being associated with an interval of time as opposed to a point in time. This helps in modeling certain real life phenomenon naturally. [StreamBase et al.] also reason about multiple streams. Both the approaches are extensions to relational algebra. The MSR approach took the algebra as the starting point while StreamBase took an existing language over the algebra – SQL as the starting point. The MSR approach consequently avoids having to rework other elements of the SQL surface. The primary language extensions through which this algebra will be exposed initially is LINQ.
What are the implications of this? Can we use the CEP algorithm to monitor real time data from the cloud and extract only the necessary data to our datawarehouse ? or am i going to far with this ?
Found at: http://www.dbms2.com/2009/05/13/microsoft-announced-cep-this-week-too/
Microsoft announced a new SQL Server version, SQL Server 2008 R2 with realy great upgrades for BI. And you can register for the CTP.
Some very interesting new features are:
- Self Service Analysis with “Project Gemini”
Build Robust Analytical Applications
-
Combines native Excel 2010 functionality with an in-memory, column oriented processing engine to allow users to interactively explore and perform calculations on millions of rows of data at lightening speeds
-
Streamlines the process of integrating data from multiple sources – including corporate databases, spreadsheets and external sources
-
Access PivotTables, slicers, and other familiar analysis features in Excel to create reports and perform advanced analysis
-
More Powerful Ad-hoc Reporting with Enhanced Data Models
Report with Ease
-
Decrease time and costs developing reports
-
Enable timely access to information to help make better decisions by empowering end users to easily design queries, reports and charts through a highly intuitive, drag and drop interface
-
Powerful and intuitive authoring and ad hoc reporting capabilities with enhanced data models
-
“Grab & Go” Reporting
Collaborate with Confidence
-
Central, secure location for IT administrators and users to publish content objects that can be broadly reused and easily customized to meet the users’ needs
-
Ensure consistency by creating and maintaining departmental content that can be accessed by the rest of the organization for building comprehensive business reports
-
Accelerate report creation by allowing end users to reuse existing components (queries, tables, charts, maps, gauges, logos) as building blocks for creating new reports
It even includes the new MDM with Master Data Services (MDS).
Read more and register for the CTP here : http://www.microsoft.com/sqlserver/2008/en/us/R2.aspx
When developing a ETL package a lot of work goes into creating test data, usually you load a file in SSIS and enrich it with data. It could be very useful when you could script this test data so you can recreate it on thy fly or share it with you colleague developers.
Using SQL Server management studio you can generate a script from the data in your tables. Go to your database, right mouse click, Tasks and click Generate scripts
Click next on the splash screen, select your database, at the script options screen set the select script data value to True, set all other properties to false.
Click next, On the next screens select all tables you want to select and click the desired output and click finish. It will now create your data script.
Service Pack 1 for SQL Server 2008 is now available for download. The service pack is primarily a roll-up of Cumulative Updates 1, 2, and 3 and minor fixes made in response to requests reported through the SQL Server community.
While there are no new features in this service pack, a notable addition from a Reporting Services point of view is the ability to configure Click Once deployment to either launch Report Builder 1.0 or the updated Report Builder 2.0, directly from Report Manager (native mode) or SharePoint. If you run Reporting Services in SharePoint integrated mode, make sure to also install the updated RS add-in for SharePoint.
Thanks Robert for the heads up.
Alexander kuznetsov of sqlblog.com has made a summary of previous posts about defensive database programming,defensive programming is an approach that is designed to ensure high quality of software by eliminating existing bugs and avoiding potential ones. The goal of defensive programming to make sure that software robustly and gracefully handles cases of unintended use. Programming defensively includes the following techniques:
- Explicitly list the assumptions that you made during development
- Whenever feasible, remove assumptions that are not essential
- Ensure that the essential assumptions always hold
- Use all your imagination to come up with cases of unintended use, trying to break your module. Incorporate these cases into your testing suite.
- Lay out your code in short, fully testable, and fully tested modules
- Reuse your code whenever feasible
Read the entire post here:
http://sqlblog.com/blogs/alexander_kuznetsov/archive/2009/03/08/summarizing-previous-posts-about-defensive-database-programming.aspx
An interesting discussing at sqlblog.com on cloud computing, what are the pros and cons? and in what time can we expect this to take a flight ?
Read the discussion here: http://sqlblog.com/blogs/denis_gobo/archive/2009/02/25/12206.aspx
Carl Rabeler from the SQL Server Customer Advisory Team, Best Practices group have released a codeplex solution automates the collection of SQL Server 2008 performance data from any or all of the following data sources ans stores the collected data into a single SQL Server 2008 relational database:
- SQL Server Profiler Analysis Services trace data
- Performance Monitor counters
- Analysis Services Dynamic Memory Views (DMVs)
This data is collected using an Analysis Services server-side trace, several Integration Services packages, a custom performance monitor collector in Management Data Warehouse, and Transact-SQL stored procedures. This codeplex project also include sample Reporting Services reports utilizing SQL Server stored procedures that correlate and analyze the collected data.These reports enable you to:
- Determine your slowest MDX queries for a specified time period
- Compare the performance of a specified query during different time periods
- Analyze a specific query to determine if it is storage engine or formula engine bound, to view the aggregations utilized, the number of subcubes and the number of partitions
- Analyze processing performance
- Correlate performance monitor counters with the execution of a specific query or processing operation
- Correlate DMVs with the execution of a specific query or processing operation
Download it here: http://www.codeplex.com/SQLSrvAnalysisSrvcs
I found the tool at the great guys from SQLCat
