 | |
What's New in Net Research Server...
version
5.03 | March 24, 2008
- shopping comparison: support XML/CSV product feeds
- spider/index: option to keep old links in index
version
5.02 | March 03, 2008
- free version: provide 10,000 pages for free version. remove all limitations
version
5.01 | January 07, 2008
- indexing: memory corruption bug
- search: improve summary generation
version
5.00l | October 12, 2007
- new features: wiki pages, metadata search, incremental indexing, extra fields in directory, popup status window, document converter script, link: and domain: search prefixes
- databases: if upgrading from v4, delete all directory and web databases, reimport, respider, and reindex
version
4.35d | July 21, 2007
- windows: fix memory allocation bug
- directory: fix subcategory count bug
version
4.31 | December 15, 2006
- NRS ODP: fix bug in deleting templates
- index: stop using up to a 200MB database buffer when rebuilding an odp or full-text index. provide override command
version
4.30 | November 28, 2006
- directory: fix bug in category filter
- index: change ranking to identify word location: occurs in text or title. Reindexing is required when upgrading to this version.
- user admin: xml api for partners to add/remove/edit urls
version
4.17 | August 21, 2006
- linux: use static libraries for demo versions. ODBC will not work but it will install more easily for different versions of libc.
- page rendering: optimize memory usage for templates with XSL transformations on XML feeds
- index: support language search
- mail: fix slowdown with relayed mail
- search: fix memory leak during peak timeout conditions
- account: new resend verify emails, use given email address in "to" field of account customer service when sending system emails
- subsql: command line tool for databases. update to latest version of database
- database: update from gigabase 3.30 to gigabase 3.51
- user admin: support uppercase, paginate user management screens, add select all, support wildcard at beginning
version
4.1 | July 03, 2006
- crawler: fix bugs in robots.txt handling, support rel=nofollow for anchored images, gif crawl bugfix, support gzip encoding from webservers
- user accounts: email verification when opening account, bugfix in image authentication,rank info on user listings
- indexer: spam prevention on words in urls,boost homepage rank,allow rank boosting per collection, external stop.words file support, allow 1 character search, wordset feature for precompiled queries
- mail: support SMTP authentication
- clustering: support duplicate removal in clustered results, allow inclusion of an invalid category id as a data source
- system: fix memory leak when using Sablotron and XML template element transformation rules
- linux: remove executable dependencies on standard C++ library and libgcc
version
4.0 | April 02, 2006
- admin: semi-automated search source testing
- templates: new tag template type for storing user bookmarks
- search: faster search performance
- database schema change: the format for document storage and indexes has been changed from v3.6. You will need to delete your web*.db files and dirindex.db files, respider, and reindex
version
3.6 | November 08, 2005
- user listings: show keyword hitcount statistics
- crawler: free domain information cache after crawl
- cached search: update hit counter in cached result xml file
- images: change ranking to benefit larger images
- crawler: fill queue with unpolite items when DNS queue maxed out
- stylesheets: display search results in larger font, remove unused CSS styles,
- templates: index elements provide extra index search feeds, odbc elements provide SQL search feeds
- server manager: manage multi-seat crawling/indexing and maintenance operations
- crawler: provide millisecond granularity for timeouts
- mail: mail forwarding rule
- crawler: provide millisecond granularity for timeouts
- search: fast index database for speedier searches: contains more common words and up to 1000 results.
- robots: obey rel="nofollow" in anchor links
- 404 error: return true 404 code even with 404 html error page
- templates: <useragent> macro, banip field to exclude element from ip ranges
- alert reports: aggregate alerts into a report, reports can be made public and other users can signup
- secure pages: place login form on page rather than javascript redirect
- demo app: review news sources
version
3.56 | October 05, 2005
- system: reduce memory usage on weblog report generation
- distributed search: fix non-sequential search result page access
- search: fix bug in phrase search
- system: fix bug in removing a Windows service
- users: fix bug in managing expired trial users
- stylesheets: display search results in larger font, remove unused CSS styles,
- odbc: allow setting of username and password of ODBC source
- accounts: implement support of multiple account options per app. user can cancel account, upgrade account. Use image verification authentication.
- image search: change ranking to favor large images
version
3.55 | September 12, 2005
- parse rules: url type always output link
- premium links: randomize and display first 5 by default, display also on search and news template types
- user listings: improved category picker and UI
- user subscriptions: yearly subscriptions and onetime subscriptions
- search result caching: save search results to file cache to speed up subsequent identical searches
- crawler: upon 401 or 403 status codes, return robot denied only after 3 such codes after one another
version
3.54 | August 26, 2005
- webserver: https support, run in secure mode with self-issued ssl certificate
- database: replication for failover and fault tolerance. one writer multiple readers
- admin: review of user listings and expired trials, with emails to remind,accept, reject,extend, delete
- directory: use meta robots no follow directive by default to prevent crawlers from crawling ODP
- web logs: show ip address report, filter by ip address, support proxied remote ip info in web logs
- webserver: ban ip feature, ban_ip.html page, 404.html page for page not found errors
- personalization: save page and email page once completed return to last page viewed
- toolbar: fix bug in downloading toolbar app
- crawler: fix bug in DNS timeouts, when timeout occurs, prevent new DNS request for 1 hour
- search forms: swap image elements for submit elements when displaying a form summary
version
3.53 | June 8, 2005
- feeds: start retrieving feeds before performing search
- mail: fix mail output CRLF characters
version
3.52 | May 24, 2005
- search results: show time taken
- alerts: fix bug preventing mailout, allow title of alert override
version
3.51 | May 12, 2005
- crawler: Fix DNS timeout, faster queue pump
- directory: export new domains to CSV directory file
version
3.50 | May 3, 2005
- news: Automatic RSS feed support
- templates: Allow links to be tracked by creating redirects
version
3.49 | April 26, 2005
- directory: import and export using a new CSV file format
- crawler: bugs in crawling out directory listings
- system: Windows memory management bug
version
3.48 | April 18, 2005
- image filetype: bug in tiff library
- setup: bug in demo installation programs for Windows
version
3.47 | April 3, 2005
- crawler: use less memory for crawl queue, pdf bug fix, zlib v 1.22
version
3.46 | March 21, 2005
- crawler: faster crawl queue
version
3.45 | February 24, 2005
- system: new memory manager
- crawler: asynchronous DNS resolver
- image search
- crawler: new crawl queue algorithm
version
3.44 | January 19, 2005
- form filter: filter search results for results with any form or search forms
- alerts: fix crash bug in highlighting
- form index: change ranking
- crawler: respect meta robots nofollow, noarchive, noindex
version
3.42 | January 4, 2005
- crawler: fix politeness algorithm and collection maximum page verification
version
3.41 | December 31, 2004
- fix pdf conversion segfault
version
3.4 | December 29, 2004
- optimized memory usage. memory pooling and conservation.
version
3.3 | December 20, 2004
version
3.24 | September 27, 2004
- stylesheets
remove popups for alerts, improve formatting
of html
version
3.23 | September 17, 2004
- crawler
accept CDATA text
- url
properties allow hookup of non search forms
to urls
- user
listings bugfix with user listings and phrase
search
version
3.21 | September 14,
2004
- user
cookie allow user cookie to be passed with userid
variable
- user
listings automatically setup category for listing
version
3.2 | September 8, 2004
- user
listings: let users submit urls to add/remove
listings from directories. users can purchase
keywords. accounts can be free or subscription
based.
- template
elements: provide parameters to urls in the
form of <param1>, <param2>,... in
parameter override or url.
version
3.12 |August 13, 2004
- collections:
override crawl distance, stay on path, and follow
cgi for individual urls
- crawler:
fix bug in maximum number of pages to crawl
per seed document, increase maximum page crawl
size from 120kb to 200kb
- document
history: always keep original crawled document
if historical versions to keep greater than
1
version
3.1 | August 3, 2004
- plugin
API: develop or use plugin components. NRS Research
Assistant is now a plugin option to NRS Enterprise.
- collections:
support one url in multiple categories for custom
directories (using ODBC import)
version
3.00.2 | July 23, 2004
- parse
rule builder: UI component for creating and
configuring parse rules.
- collections
report: Run report on a collection to view and
edit status of each url
associations: Create content associations to
map content from one set of templates to another,
following rules.
- xsl
editor: Edit XSL templates using UI component
system: Make verbose log setting sticky
- feeds:
Add option to provide user ip address to feeds
and searches
version
2.99.15 | June 30, 2004
- crawler:
Bug in robots.txt handling when redirected to
invalid robots file.
version
2.99.14 | June 17, 2004
- toolbar:
New Internet Explorer toolbar feature. Ties
to NRS application. Provides news, search, access
to alerts and bookmarks, popup blocking, and
personalized searchsets.
version
2.99.12 | May 22, 2004
- database:
turn off linux kernel page caching on database
access for better memory performance.
version
2.99.11 | May 21, 2004
- library:
What's new feature.
- ODBC
import: Star rating on urls.
version
2.99.10 | May 18, 2004
- ODBC
directory import/export: use your own directory
instead of ODP. Import ODP and customize it.
- Java
client database library: access NRS databases
over Tcp/Ip using Java or .NET.
version
2.99 | April 4, 2004
- templates:
more common elements to help customize entire
apps from one XSL template
- directory:
include and exclude URLs as part of ODP import
- directory
import: accomodate new issues in RDF file format:
duplicate category relationships, incorrect
category paths, and a new topic tag
- crawler/indexer:
PDF document support
- cookies:
ability to set cookies from XSL template by
sending s-c=name:value or s-t-c=name:value in
URL
- personalized
search: select categories of interest to boost
ranking of matching documents
version
2.98 | February 25,
2004
- admin:
improved help
- directory:
import ODP manually, instead of automatically
upon first time program startup
version
2.97 | February 17,
2004
- spell
checker: you can assign a spell checker to a
template. Supports most languages.
- collections:
import/export documents, index and crawler now
supports documents residing in multiple categories
- queries:
boolean query support
- administration:
UI improvements, drop down boxes limit selection
choices for more intuitive management
- ODP:
fix ODP import problem causes by invalid ODP
category in RDF dump
- performance:
query performance doubled
version
2.96 |January 26, 2004
- account:
account template for user login, subscriptions,
signup, e-commerce
version
2.95 | January 19, 2004
- crawler:
critical bugfix for domains with no periods
- news:
better support for RSS/XML news extraction
version
2.94 | January 15, 2004
- site
search: UI bug: cannot unset stay on site
- system:
update to Gigabase 3.11
- indexer:
added option for compact index, and word stemming
- collections:
collection settings controlling crawler now
also used for indexing eg stay on site, followcgi,
max pages,..
version
2.93 | January 12, 2004
- demo
application: update news and search sources
- site
search: fix bug in setting site search root
version
2.92 | January 6, 2004
- crawler:
stayonsite now includes subdomains
- site
search: new NRS Site Search product
version
2.91 | December 16,
2003
- database:
critical bug fixed in database object reallocation.
This fix prevents possible database corruption.
- system:
upgrade to Gigabase 3.07. Incorporate OpenSSL-0.9.6l
- https
support: crawler now also crawls https documents
- cookies:
crawler can include user defined cookies for
site crawling and searching
- virtual
domains: you can specify a host for a template
or application. This allows for virtual domain
hosting with 1 ip.
version
2.90 | December 2, 2003
- directory
search: improved performance on phrase search,
and rank boost to juxtaposed search terms in
'AND' search
- system:
upgrade libraries to Gigabase 3.05, expat-1.95.7,
pcre 4.4, zlib 1.2.1
version
2.89 | November 5, 2003
- directory
search: drill-down refine search by category
feature
- library:
fix bug in crawling and indexing libraries with
-nodmoz flag
- directory
search results: add remove duplicates feature
version
2.88 | October 24, 2003
- linux:
use gcc 3.22, and associated pthreads, stdlib,
and glib libraries
- windows:
use STLPort 4.5-1009
XSL templates: more compact search results presentation
- page
streaming: add stream tag for custom proxy page
flushing
- email
folder detail: there is a new UI feature (a
link called show detail) that shows the content
of the message.
- error
duplicates: option to remove duplicates from
collection searches
- popups:
remove popups for search source / news source
detail
version
2.87 | October 7, 2003
- alert
crawling: add new variable to control politeness
during alert crawling
- XSL:
provide a way to retrieve partial template overrides
using .xsl?partial=1
- XSL
templates: more compact search results presentation
- pop3:
you can retrieve your new mail, there is no
option to leave the mail on the server
email folder detail: there is a new UI feature
(a link called show detail) that shows the content
of the message.
- error
information: on XSL templates. You now get a
proper error message on Linux.
changes to default settings:: pop3 and smtp
off by default, porn filter on dmoz off by default,
dmoz update interval now 14 days
- serve
new filesystem types:: you can use the following
extensions: htm,html,shtml,txt,c,c++,pl,cc,h,txt,css,js,gif,png,jpeg,jpg,ico,bmp,tif, tiff,wav,mp3,au,aif,
mid,ra,ram,rm,rpm,mpg,mpeg,asf,asx,mov,avi,doc,rtf,map, ps,ai,eps,swf,dir,dcr,dxr,hlp,
chm,class,pdf,tar,tgz,gz,jar, cab,gzip,zip,ppt,pps,xls,vrm,vrml,wrl
For files with these extensions it first looks
in the filesystem to serve it. Then for certain
filetypes like htm,html,txt it will see if it
can serve it from a template.
- built
in subscription payment support:: The library
template now has built in support for subscription
payment. When you sign up, if trial period is
0 in system settings, it asks you for basic
signup info. If trial period is set, it asks
you for more info such as company, phone,..
If you add the variable '&subscribe=1' then
it also goes through the payment page. There
is support for iBill and PayPal, the only processors
out there that support recurring subscription
payments without a merchant account. By setting
variables in drawsubscribe you can enter your
iBill codes, your PayPal codes, and select which
payment options you want to use. Also there
is a built in privacy and terms and conditions
pages with variables to control the application
name and company name.
version
2.86 | September 9,
2003
- alerts:
control max alerts per user account and default
max alerts
- reindex
after crawl: new flag to automatically reindex
after a crawl
- full-text
indexing: index words containing digits up to
6 characters, or 15 characters if the word contains
dashes
version
2.85 | September 4,
2003
- login:
return to source template after login, if any.
- templates:
added related links feature to templates
- system:
remove DNS thread cleanup algorithm, fix Linux
issues
version
2.84 | July 18, 2003
- save
to library: save alerts, news subscriptions,
and pages to a library folder. Setup the library
folder to be further crawled and indexed.
- improved
page titles
- database
locking: fixed sporadic deadlocks
- libraries:
updated gigabase to version 2.96, pcre to version
4.3, sablotron to version 0.98, and expat to
version 1.95-6
version
2.83 | May 24, 2003
- alert
and custom pages: bug fixes
version
2.82 | May 16, 2003
- full-text
indexing: use intermediate file for faster indexing
- crawler:
robots.txt bug, faster failed connection timeout,
collection reset feature
- templates:
highlight bug, phrase search bug, category exclude
filter feature
- mail:
smtp mail relay feature to get round issues
with reverse DNS checks and virtual hosts
- proxy:
distributed search queries, proxied requests
for directory templates
version
2.81 | May 6, 2003
- full-text
indexing: improved international character indexing,
graceful no memory logic
- templates:
new only in category search
version
2.80 | April 24, 2003
- crawler:
better detection of crawlable file types, memory
issue with large crawl
- search:
better summaries
version
2.79 | April 21, 2003
- crawler:
new limit of 10 million page max
- full-text
indexing: improve ranking, support for 3GB machines
under Windows, better international character
handling
- search:
return normalized rank to 100
version
2.78 | April 13, 2003
- crawler:
politeness bug
- full-text
indexing: improve title ranking
version
2.77 | April 09, 2003
- full-text
indexing: improve ranking, fix popularity algorithm,
fix crash, add site:domain feature, double index
words with +,-,_
version
2.76 | April 07, 2003
- library:
fix javascript bugs for IE in popup links
- universal
search: more sources against web index
- database
backup: disable database compaction to fix corruption
bug
- custom
page: bumped max sources from 15 to 50
- crawler:
immediate crawl start/stop, save crawl queue
between crawls. fix freeze bug in saving counts.
- full-text
indexing: exclude local anchors from popularity
index
version
2.75 | April 03, 2003
- full-text
indexing: optimize memory footprint, fix bug
in popularity matrix
- crawler:
optimize politeness handling
version
2.74 | March 28, 2003
- crawler:
optimize crawl queue
- collections:
change defaults and add stay on site
version
2.73 | March 26, 2003
- full-text
indexing: Addressed performance issues in scanning
urls
- directory
xsl template: Modified display of newsgroups
and editors
- news
xsl template: Sort news sources by title
- directory
crawl: Only crawl directory templates marked
as full-text if directory crawl option enabled
- alert
email: Allow CC and BCC
- edit
alert: Add delete button
- alert
update schedule: Make alert updates work on
own schedule independent from web crawl
version
2.72 | March 24, 2003
- cross-platform
databases: Fix compatibility bug swapping databases
between Linux and Windows
- collections:
Fix max depth and max distance implementation.
- cc
email alerts: Allow email alerts to be cc'ed.
- exact
phrase: Support exact phrase search.
version
2.71 | March 10, 2003
- dmoz
rdf path: Fixed dmoz rdf download path. Changed
to : http://rdf.dmoz.org/rdf/
- crawl
bugs: Fixed crawl bugs to do with crawl queue,
site spidering.
- full-text
index: Make more compact indices, optimized
btree index creation,
- exact
phrase: Support exact phrase search.
version
2.7 | February 11, 2003
- web
index: Crawl and index ODP content
- demo
application: Over 350 search engines and 100
news sources built in.
version
2.5 | November 22, 2002
- newsletter
signup: Signup to newsletters in ODP categories
- category
mailer: Send mail to ODP category members
- history:
View page differences over time
- cached
pages: View cached document with highlighting
- application
building: Build applications with custom mail,admin,dmoz,search,news
- admin
UI: Streamline UI with popups and easier navigation
- dmoz
full-text index: Dmoz full-text index and metadata
- news
subscription: Subscribe to news by mail with
keyword filter
- news
page: News template.
- max
page crawl size: 120KB
- search
detail: Provide names and descriptions to searches
- custom
search page: Build your own search page
- search
alert: View results by source
- automatic
backup: Scheduled backups
version
2.4 | July 5, 2002
- adminpeer:
Support admin security by ip address and/or
username
- mail
smtp: Fix bugs in SMTP protocol
- nodmozcrawl
option: Disables related searches on dmoz templates
- search
descriptions: Searches can have a popup with
description, title, and url
- error
reporting: Automatic reporting of exception
errors to log file with stack trace
- color
coding of template elements: Hidden elements
are shown in red.
- user
form crawl: User forms are no longer refreshed
as part of crawl
- syncforms
commandline: Forces synchronization of forms
in templates and form database
- admin
copy: You can now copy elements in between templates
- UI
sorting: Lists of searches and groups now sorted
- parse
rule error reporting: Admin interface provides
error diagnostics on parse rules
- parameter
override: You can specify the query terms in
the parameter override. Fixed minor bugs.
- parse
rule: New attributes to filter out header,footer,
and bad results. Support for image link extraction.
Support for appending to links.
- autorefresh
of snippets: Snippets and XML feeds are now
automatically updated before cache expiry
- new
stealth option: A new stealth option for obeying
robot instructions only in case of dmoz URLs,
not for user added URLs
- admin,mail:
New template options
- template
pathnames: Organize applications and groups
by URI pathname
- tab
interface: Improved tab interface in applications
- xsl
override: You can now override XSL template
blocks and inherit rest from base template
- Gigabase:
updated libraries to version 2.64
- Sablotron:
updated libraries to version 0.95
version
2.3 | April 25, 2002
- form
encoding support: Support for forms that use
multipart/form-data
- external
image serving: images can be served through
HTTP from the root application directory or
any sub-directory
- stealth
option: use common Mozilla user agent for web
requests and ignore robots.txt. By default this
option is turned off, and is only recommended
for personal/intranet use.
- application:
you can build applications out of templates.
By also specifying a group, the templates within
an application can be selected through a tab
interface and drop down options.
form parameter override: you can also specify
new parameters for form definitions in searches
- Gigabase:
updated libraries to version 2.60
- Sablotron:
updated libraries to version 0.90
- xml
aggregation: xml feeds are now rendered into
html by default
- save
page: save page to a folder
version
2.2 | March 14, 2002
- database
API: new SDK to access database through perl,
C++, Java
- settings:
settings are saved in settings.db
- porn
filter: filter out porn by default from dmoz
- dmoz
netscape topic: removed netscape topic during
import
- forms:
forms with more than 20 inputs not imported
- dmozimport:
fixed duplicate category id bug
- email:
email to a friend
- sort:
sort functionality for search results
- -path:
added -path option to specify executable current
directory
- -log:
added logging of web requests and UI to view
stats
-daemon: fixed unix daemon support
- xml
encoding: remap macintosh quotes to normal ones
- searchbox:
removed extra searchbox from search template
- backup:
backup will now defragment and compactify database
- rank:
added rank in xml for search results
- content
cache: bug in xml generation from cached searches
- search
parsing: fixed crash bug in numbered hits extractor
- adding
search to template: fixed bug where crawl of
search could take up to 1 minute
- msxml4:
automatic support for msxml4 parser as well
as msxml3
- top
categories: some categories where not being
displayed
- xml
for sitehits: text in url and hurl tag now have
http:// appended for consistency
- Gigabase:
updated libraries to version 2.57
- Sablotron:
updated libraries to version 0.82
- metasearch
results: correct spacing bug in title and description
text
- and
not operator: fixed fatal error
- page
not found error: returns http code 404 instead
of 405
version
2.1 | January 29, 2002
- logging:
a new command-line option 'weblogdir' lets you
specify a directory to which all web requests
are logged. The log format follows standard
weblog guidelines.
- form
crawling: form crawling has many bugs fixed.
Amongst them is the correct recording of option
values of html select elements.
- dmoz
roots bug: bug fix that prevented the proper
overriding of the top-level categories of a
dmoz template.
- form
post: bug fix for proper posting of forms with
no submit buttons.
- parse
rule: bug fix in parsing algorithm involving
labels, prefix/suffix detection, and different
occurs values.
- upload:
new upload feature in admin interface let's
you upload changed xsl templates through http
file upload.
- sort:
you can sort columns in lists in admin interface.
- robots.txt:
NRS is now robots.txt compliant. It will always
query for robots.txt before crawling a page
on a site or retrieving search results from
a page.
- admin
lists: The admin interface exposes the list
of dmoz categories, dmoz urls, mail accounts,
and crawled form definitions.
- picker
ui: The admin interface lets you pick categories
for the roots template field, and lets you pick
search forms for the add search template functionality.
- search
result relevance: Search engines returning no
results are placed at the end.
- dmoz
autosearch: New functionality lets you autosearch
the found search engines using the current query.
- new
operator: The new and-or operator searches categories
using 'and' and searches sites using 'or'. This
results in better search engine autosearch results.
version
2.0 | December 6, 2001
- admin
interface: A new admin interface exposes search
engine functionality and lets you build templates
based on the default system templates.
- metasearch:
Build your own metasearch page from search engines
you choose.
- crawler:
The crawler is now built into NRS. No need to
download databases anymore.
| | |
