Discussion in 'kdb+' started by itfin, Apr 14, 2008.
《Q for Mortals 2.0》的Word版本，适合打印。。
2、Kdb 附有SQL92接口，for pure SQL functionality.
Database optimization techniques in Kdb could help ease the enterprise data crunch
DATA ANALYSIS IS getting out of hand. As enterprise data warehouses grow rapidly, the requirements for data analysis to produce timely, meaningful results seem to be hitting a brick wall.
IT SOLUTION SEARCH
With so many touch points for customers to interact with the enterprise, the demand for fast data-mining capabilities in applications such as e-CRM (customer relationship management) is quickly growing. (Look next week for the InfoWorld Test Center Research Report on CRM and the importance of CRM analytics.)
With data demands showing no signs of easing up, it makes sense that a new way of thinking and processing records may be just what your IS department needs. And that's why a company such as Kx Systems and its Kdb database is so interesting.
Kdb is a high-performance relational database with optimized storage and retrieval that enables analysis speeds as much as one thousand times faster than those in traditional databases.
With benchmarks of a million random-write updates per second and reads of tens of millions of record aggregations per second on data sets billions of records deep, Kdb should raise the brows of financial and e-CRM application developers. Kdb is ideal for accessing large volumes of historical data and capturing real-time transactional events.
Rather than continue to build optimized look-up routines on old database paradigms, Kdb gains its speed advantages by shedding the conventions of traditional, row-based databases in favor of optimized, column-based storage.
Although record-based storage works well for distributed parallel tasking, Kdb's inversion process allows entire columns to be aggregated in a single call rather than row by row.
Furthermore, Kdb has an attractively small footprint -- the download is only 150KB -- and takes up little memory, which makes it interesting for small devices such as PDAs.
The ability to quickly analyze massive amounts of time-series and real-time data makes Kdb well-suited for areas such as OLAP (online analytical processing) and OLTP (online transaction processing).
Whereas OLAP typically demands large table manipulations and preaggregation tricks such as denormalization and indexing to speed processing, Kdb makes it possible to reduce the number of bulk OLTP system-write operations from thousands to no more than the number of columns in your table. As such, Kdb boasts the capability of processing several million OLAP records each second.
Kdb's open-standards approach offers solid connectivity through JDBC (Java Database Connectivity) and ODBC, as well as its native KDBC (Kdb Database Connectivity), HTML, and XML. Kdb is available for a range of platforms, including Windows NT, Windows 2000, Linux, Solaris, AIX, HP-UX, and FreeBSD.
Although Kdb includes a SQL92 interface for pure SQL functionality, it also supplies its own KSQL, which advances query capabilities, adds time-series functionality, and natively takes advantage of the K engine.
I was impressed with the K development platform's well-integrated set of components, which included an interpreted, financially optimized programming language; data management system; primitives; and GUIs under one roof.
K offers good sort and search capabilities; is extremely readable, portable, and powerful; and uses a very small vocabulary. In addition, it offers strong distributed computing and integration into IS environments with client connections for C/C++, Java, Visual Basic, Excel, and more.
Changing to a new platform will, of course, cause some growing pains. But learning to use K and KSQL should not be difficult for anyone comfortable with standard SQL.
The most time-consuming aspect will be learning to write good extensions for time-series analysis. But even here, experienced SQL programmers should have their bearings in KSQL within a week or so.
The software can be downloaded from www.kx.com, along with the K development environment and the manuals.
Now, I'm not recommending that you pick up camp and move your Microsoft, Oracle, Informix, or IBM-stored assets. In fact, only a limited number of SQL system migrations have been successfully ported to Kdb.
But if you are writing complex time-series queries and are waiting on results, Kdb can provide performance enhancements as a supplemental application server that can offload some of the strain from your enterprise databases.
As a project to build a near full-scale application in q, the following articles work towards implementing an algorithmic strategy in q:
Building qx, a test exchange ... read more
A market maker for qx ... read more
The participation strategy specification ... read more
Implementing the strategy, part 1 ... read more
Implementing the strategy, part 2 ... read more
安装了odbc.exe,在ADO下面用它网站上的odbc.txt 上的说明的 连接串 "Provider=MSDASQL.1;DRIVER=kdb+;DBQ=localhost:5001;UID=usr;PWD=pwd;"连不通。
1）没有设置好port。默认情况下q没有打开port，你必须在启动q时带参数（q.exe -p 5001 ,p为小写字母)或启动后输入命令（\p 5001）来设置port。
2）启动q时加上"-U "或“-u ”参数：
q -p 5001 -U c:\q\w32\qusers
q -p 5001 -U c:\q\w32\qusers 中的“\"可能要改为"/"
在ADO通过ODBC连接后,发现Q语言的那些查询语法, 比如 select [a] [by b] from t [where c] 等,不能通过ADO的执行.看来ADO只能执行标准的SQL,Q语言和SQL有分别,不能这样用.
This python interface to KDB is modeled after the java interface for KDB. It interacts with the binary interface for the KDB Database.
It is currently BETA quality. It works, but needs some rigorous testing.
It allows you to send queries and data to the database and receive data back.
Everything is implemented in a single python file so there is no compiling necessary. just drop the q.py file into your project and you can quickly query the database with just a few lines of code.
conn = q.q.('server', 5000, 'username')
result = conn.k('select from trade')
Interview with Arthur:http://mags.acm.org/queue/20090203/?pg=12。