DBSight Index Configuration Upload/Download

Submitted by chris on Wed, 2006-09-13 20:45.

Now you can easily port existing index configurations to another DBSight instance. Just choose download on the dashboard, the index's defintion, and related templates will be zipped for downloading. When uploading, it'll just be there and you don't need to restart.

This will be grealy helpful for deploying index created in development environment to production environment.

It's also very helpful for multi-server mode. You can visit for more details. You can dedicate one DBSight instance as server for indexing process, and other several instances as clients to process searching requests.

What's about DBSight?

Submitted by will on Mon, 2006-09-04 00:36.

DBSight uses Lucene, so it inherits Lucene's fast performance. You may think Lucene's API looks very simple, why not use Lucene directly? But when really used in database-related search, you need to

  1. Connect to databases (Connection pool?)
  2. Select the content (What if the content is in several tables?)
  3. Save as an full-text index (How to analyze the data?)
  4. And search ...
  5. And render search results

Very simple? No! Far from enough!

  • You need to parse user's query into Lucene's format.
  • You need to maintain the search when you are indexing/crawling the data.
  • You need to determine what's already indexed and what's not.

MySql is giving Java.lang.outOfMemoryError on large ResultSet

Submitted by chris on Sun, 2006-04-30 18:42.

MySql is always giving Java.lang.outOfMemoryError on large ResultSet

After some debugging, on 5.0.0 beta,in, function fetchRowsViaCursor, there is a parameter fetchSize

 protected List fetchRowsViaCursor(
  List fetchedRows, long statementId,
  Field[] columnTypes, int fetchSize)

The fetchSize is not really used in

  while ((row = nextRow(columnTypes, columnTypes.length, true,
    ResultSet.CONCUR_READ_ONLY)) != null) {

So I changed it to this, recompile, and it works!

  for(int i=0; i< fetchSize&&(
    (row = nextRow(columnTypes, columnTypes.length, true,

Lucene Date Range Search

Submitted by chris on Thu, 2006-04-13 11:17.

In DBSight 1.2.7, there will be range search available:

So if you want something like this:

 select *
 from projects
 where content like '%keyword%'
 and created_at between to_date('2005/01/01')
    and to_date('2005/06/01')

For Date Range Query, if "created_at" is a Keyword, "Date" or "DateTime" or "Time" type, the query grammar will be like this, search with query:

 keyword created_at:[2005/01/01,2005/06/01]

Lucene api changes for numeric range search

Submitted by chris on Mon, 2006-04-10 10:12.

From Ken,

 Chris, the release notes from DBSight 1.2.6 say that
 it includes Lucene 
 2.0RC1.  That hasn't even been tagged, as far as I
 can see.  You 
 certainly know more about the quality of the Lucene
 source tree than I 
 do, but it really concerns me putting development
 snapshots into 
 production here, especially with the problems I've
 had with regressions 
 in DBSight.  Perhaps you need a public beta process
 or "unstable" 
 releases to introduce new functionality.

Thanks for letting us know your concern! It does make sense that our too aggressive approach may cause problems to production environment. news

Submitted by will on Tue, 2006-03-21 20:31.

Steve Mallett found our site

When we got his email, we thought, gee, spam emails are so intelligently targetted these days! And all of us moved the email to trash. :)

Sorry and thanks a lot, Steve!

SQL to search common Bug system?

Submitted by will on Mon, 2006-02-06 22:46.

I am interested to know if anyone is using DBSight to search their existing bug system.

Usually, bug system is also a good knowledge base. It has solutions, work-arounds, etc. It'll be nice if the bug system is easy to search, without the complicated "Advanced Search", which is done by SQL, slow and no ranking.

DBSight is an ideal solution to search on bug systems. Actually, DBSight's testing environment is a database of a high-volumn constantly changing bug system.

It'll be nice if anyone can send us the SQLs and templates he/she uses on some popular bug system, like Bugzilla, Mantis, etc.

Compared with MX Site Search

Submitted by will on Thu, 2005-12-15 14:01.

My friend send me a link.

It's a product for searching in database also. It has a nice UI for you to develop. But it has limitations:

  1. No customization on document ranking, result filtering, multiple language support, etc
  2. Only for MySQL, Postgres, and Access
  3. Only for php, coldfustion, or IIS
  4. No link between multiple tables

If anyone has tried it, let me know the performance with large data volumns.

DBSight has all the features MX Site Search claims

  • Search in multiple tables
  • Easily to configure search engine

Can Lucene be Used To Substitute Real Database?

Submitted by will on Tue, 2005-10-25 13:09.
 > Can Lucene to be used in place of mysql so that
 > website visitors can input data that will in turn
 > inserting row into Lucene just like mysql db?
 That's a bad idea. Lucene lacks a real update (you need to delete and
 re-add) and also sees everything as a string, even numbers. So although
 it's technically possible you don't want to do it.

First of all, just using Lucene to replace rdms is quite possible in some specific cases.

In addition to updating and string/number issues, Lucene also lacks many rdms functionalities. One of them is aggregation functions like SUM(), or "group by".

Bad Search is One of the top 10 web design mistakes

Submitted by will on Wed, 2005-10-05 10:37.

One of the top 10 web design mistakes:

5. Bad Search Everything else on this list is pretty easy to get right, but unfortunately fixing search requires considerable work and an investment in better software. It's worth doing, though, because search is a fundamental component of the Web user experience and is getting more important every year.

Here is a very good article on site search design.

Good to know DBSight is designed to create a site search in hours!

XML feed