***** TWISTER NewsServer ***** Copyright (c) 1999-2007 Highwinds Software LLC All Rights Reserved. www.highwinds-software.com software@highwinds-software.com 1.866.NNTP.119 ------------------------------------------------------------------------------- Twister 2.1.394 - Added deeper retention to History. ------------------------------------------------------------------------------- Twister 2.1.390 - Added required PostFilterErrorAction configuration key/value where the value can be either 'Accept' or 'Reject'. This determines what action to take when the external postfilter is unreachable due to either the program terminating or failing to respond to read/writes. Accept was the previous action taken in all cases. Reject causes the user's article posting to fail completely. - Added Terabyte directive for spool sizes. ------------------------------------------------------------------------------- Twister 2.1.1.380 - Semantical fix with xover, xover # was interpreted as #- that has been corrected - Eliminate/reduce article loss during server shutdown. ------------------------------------------------------------------------------- Twister 2.1.1.371 - XOVERs are now correctly bandwidth-limited for MaxBytesPerSecond and FeedMaxBytesPerSecond. - The -strict option will prevent users who send bad commands from being disconnected. - More robust handling of huge overview lines. - Bug fix: Spools larger than 32Gb are now handled correctly when starting with -nocreate. - Added -veritas flag which enables VXFS Direct I/O if -direct is also set. ------------------------------------------------------------------------------- Twister 2.1.1.370 - Removed message marking debug output from syslog. - "Terabytes" now recognized in spool objects. - Improved shutdown stability. ------------------------------------------------------------------------------- 2.1.1 - Added a new directive called XrefMaster. Use of this directive allows only "trusted" host names that perform article number generation to store articles. The directive XrefMaster takes one or more wildcard arguments. At least one of these arguments have to match the host field in the Xref: header field when the server is set to XrefAction Pass mode. A match failure in parse mode forces an article rejection with code 0x. This directive would apply to cyclone.conf, typhoon.conf, twister.con, and tornado_be.conf. -Fix a memory leak that can cause exhaustion of all RAM use over long periods of running time. -------------------------------------------------------------------------------- 2.1.0 - New memory model drastically reduces the memory footprint - -nonblockingclose option to attempt to recover thread resources when closing reader connections. This is useful when a server is receiving many multiple connect/disconencts. - Large spools now supported, Spool sizes may now be express in "Megabytes", "Gigabytes", and be up to 2 Terabytes in size. - All XOver lines are now individually validated for correctness - Support -keyinfo option in validate - Fixed URL parsing error when variables go after a URL - Fixed redundancy in the expanded/collapsed list -------------------------------------------------------------------------------- 2.0.7 - Connect limit demoted from Error to Info (-detailed to see it) - Enhanced logging for syslog messages that displayed (m) for non-Solaris Operating Systems - Crashing when syslog > 2G fixed (Linux only) - Empty syslogs no longer emit critical errors -------------------------------------------------------------------------------- 2.0.6 - Addition of "Megabytes", "Gigabytes" and "Terabytes" for article spool sizes (Terabytes not currently used, it is for later addition of large spools) - CIDR code optimized - ChainMessageIDRequests True|False (default true) wether or not message ID requests will be passed back to a chaining host - Chaining timeouts loosened up for loaded backends. - -nohistory flag added to trust all backend content - -strict flag for strict/narrow NNTP interpretation - cat.cat file corrupt bug fixed - large spool supported ------------------------- ***UPGRADE TWISTER*** To UPGRADE to Twister 1.2.2 or 2.* from Twister 1.2.0, follow these procedures: 1) untar the distribution in a new directory. 2) Shut down Twister. 3) replace these files from the new distribution: bin/twisterd bin/resolver bin/validate bin/wrapper bin/mailslave tools/activeCreate 4) Remove tools/activeConvert 5) Replace the templates/ directory tree 6) Add new directives to twister.conf as listed in ***CHANGES*** below. To UPGRADE to Twister 1.2.0 from Twister 1.1.1, follow these procedures: 1) untar the distribution in a new directory. 2) Shut down Twister. 3) Replace these files from the new distribution: bin/twisterd bin/resolver *** IMPORTANT (protocol change) *** bin/validate bin/wrapper bin/mailslave tools/activeCreate 4) Remove tools/activeConvert 5) Replace the templates/ directory tree 6) Add new directives to twister.conf as listed in ***CHANGES*** below. To UPGRADE to Twister 1.2.0 from Typhoon 1.2.2, follow these procedures: ***NOTE*** You cannot run Typhoon and Twister at the same time, but you can revert from Twister back to Typhoon if necessary. Please contact software@highwinds-software.com for downgrade instructions. IT IS VERY IMPORTANT THAT YOU CONTACT software@highwinds-software.com IF YOU NEED TO DOWNGRADE. UPGRADE INSTRUCTIONS: 1) untar the Twister distribution in a new directory. 2) cd to the Twister install area 3) Replace the overview directory with a symlink to the Typhoon overview directory. 4) Replace the upstream directory with a symlink to the Typhoon upstream directory. 5) cd to the bin directory 6) Copy your Typhoon bin/start.conf over the Twister start.conf 7) Replace resolver ***IMPORTANT (protocol change)*** 8) If you use the -port option in Typhoon, replace it with -nntpport 9) cd to the install directory 10) Copy twister.conf-EXAMPLE to twister.conf 11) Copy your spool, overview cache, and chaining definitions from typhoon.conf to twister.conf, updating paths as necessary. DO NOT CHANGE the SpoolNumber OR CacheNumber when copying! 12) Update any defaults in twister.conf that you had changed in typhoon.conf 13) Copy your feeds.conf from your Typhoon installation. 14) Disable any automatic jobs that might restart Typhoon. 15) cd to your Typhoon install area 16) Run bin/stop. 17) cd to the Twister install area. TO CONVERT THE ACTIVE FILE: 1) cd to the etc/ directory 2) copy the Typhoon etc/active and etc/active.idx files to this directory. 3) run "tools/activeTool -upgrade -from active -to active" to produce the new active files active.table, and active.names. 4) cd (back to the Twister install area). 5) run bin/validate to make sure you aren't missing anything. 6) run bin/start 7) After startup, there will be a human-readable "etc/active.text" file. ------------------------- Twister 2.0.5 -> Configuration Run-Time Reloading added. Can now reload feeds without restarting the server. Affects the following threads: -> Writer Thread -> Accept Thread -> HTTP Accept Thread -> HTTP Reader Thread -> Standardized Naming for all download distribution oraganizations/additions. -> Command Line options available in the .conf file. -> Back end support for Advanced Chaining. -> "-maxpendconnect" command-line flag added. ------------------------- Twister 2.0.0 -> Fully compliant to published and de-facto Internet standards for News server software (USENET and USENET II). -> Performance and scalability to "carrier-class" levels: Thousands of Browsers, Thousands of Groups, Millions of Articles, Billions of Bytes per day, Terabytes (Trillions of Bytes) of storage capacity. -> True 100% availability. No need to "throttle" or have downtime to "expire". -> "Virtual Server" ability leverages hardware resources by allowing multiple "virtual" News systems with private independent newsgroups and subscriptions. This feature alone provides ISP's with the ability to host multiple customer's news services on the same hardware without additional investment. -> Cached architecture provides more efficient resource utilization. -> Full and Partial bi-directional NNTP feeding. -> Customizable Anti-SPAM filters for eliminating unwanted content. -> Customizable POST filters for eliminating unwanted locally-authored content and enforcing USENET policy on user base. -> Fine-grain content control via numerous filtering and partitioning directives. -> Detailed Inbound, Reader, and Content statistics suitable for billing. -> Carrier-class scalable NEWNEWS implementation. -> Optional Real-time statistics for monitoring and auditing individuals. -> NNTP performance analysis tools useful in diagnosing bottlenecks at customer sites. -> Powerful disk space allocation directives eliminate disk space shortages and allow flexible disk resource allocation. -> Multiple data compression algorithms for optimizing disk space consumption. -> Customizable program-based authentication. External authentication programs can maintain database connections (to Radius, Kerberos, NIS+, etc.), use time-of-day, system load, or any other data at its disposal to dynamically, allow, deny, or implement different policies of access and authentication at any time. -> Tight control over user authentication and article generation. -> Built-in enterprise-wide fault tolerant article number synchronization. -> Built-in enterprise-wide automatic active file content synchronization. ------------------------- Twister 1.2.2 -> The multi-processor version of Twister is now twisterd (previously it was twisterd_mp). -> The uni-processor version of Twister is now twisterd_uni (previously it was twisterd). -> The distribution directories have been simplified. -> All sample Perl scripts are now located in the new samples directory. -> All authmaster-related files have been moved from the etc directory to the new tools/auth directory. -> Solaris 8 support. -> Faster spam filtering. -> The new -fastfilter option causes Twister to send articles to the spam filter program via shared memory rather than the slower STDIN/STDOUT. -> A new Openwave::FilterAPI perl module has been added which makes it easy to access the articles in shared memory. -> Support for "real" threading in template overviews. -> Previously, only a subset of all the articles were threaded,which resulted in some articles not appearing in overviews. Now all articles in a group are threaded. -> A new thread cache is used to cache the threading information and avoid having to re-thread the entire group on each request. -> ovEncodedArtNums template variables used to 'mark all as read' in threaded overviews. -> MaxConnectTime can be set to limit how long connections remain open. -> GroupAllowFile and GroupDenyFile can now be overridden by the auth program. -> fetchArticle template option allows article bodies to be displayed in overviews. -> 'calc' function for performing integer arithmetic in templates. -> Support for previewing HTTP posts before submitting. -> Added empty_group_subject_line, empty_request_subject_line which are localization variables that specify what message to display for overviews of groups which are empty. -> A new set of sample templates has been added which demonstrate the the preview and fetchArticle features. The new templates are located in the templates/ShowArticles_Frames directory. BUG: The standard error message template variable 'message', was incorrectly named 'errmsg' in some cases. All occurrences of 'errmsg' in templates should be changed to 'message'. Though 'errmsg' is now obsolete, it is still set by Twister so that existing templates which use 'errmsg' will not break. However, it will be removed in future versions of Twister. BUG: auth override of Subscription/FilterSubscription is now properly handled. BUG: auth override of MaxBytesPerSecond is now properly handled. BUG: Templates now correctly display date variables for articles dated 1-1-2001. BUG: Post pages were refreshed by the server such that clicking the 'Back' button would take the user back to a blank form. Fixed by adding a cacheTimeout template variable which instructs the server to not refresh the page. BUG: Removed no-cache headers from HTTP '304 not modified' responses which was causing some browsers to refresh images on each request. BUG: Occasional crashes during shutdown corrected. ------------------------- Twister 1.2.0 -> Active bandwidth limiting for automatic response to abusive users -> Message sorting by thread, date, and subject in HTTP interface -> More efficient DNS resolution -> Reduced numbers of threads for better performance above 1000 users -> HTTP dynamic-categorizations with virtual server support. -> HTTP administrator interface for message and newsgroup control. -> Arbitrary date formatting in XML templates with the formatDate function. -> bin/reload forces upstream/ directory retry flushing -> Faster memory allocator for twisterd_mp -> Large (>2Gb) file support for all logfiles in Solaris. -> newsrc-like tracking of read messages in HTTP interface: "message marking" -> Support for disabling NNTP or HTTP by setting "-nntpport" or "-httpport" to zero on the command line. -> Support for reply-counting in threads with the artNumChildren variable BUG: IE 5.0 HTTP POST problems resolved. BUG: Article retrieval by message-ID fixed in HTTP interface. BUG: Better logging of malformatted URLs due to template errors BUG: Better startup, shutdown, and error-handling in authmaster/authslave. BUG: Connectivity loss to upstream or chaining master handled gracefully. BUG: Fixed handling of NEXT and LAST in chaining boundary cases. BUG: Fixed handling of UUEncoded and MIME-multipart posts in HTTP interface. BUG: Mixed-case group names now handled correctly in NNTP. ------------------------- Twister 1.1.1 -> AllowActiveControl directive for web-based group creation/deletion. -> Bandwidth limiting for HTTP. -> Support for If-Modified-Since and other conditional HTTP requests. -> Requested hostname gets passed to auth program. -> POST and GET variables for all requests are passed to the auth program. -> Better handling of upstream servers that throttle for hours. -> HTMLEncode() function now in XML. -> Support for multi-level sorting and sort-direction in XML overviews. -> Emission of HTTP proxy caching headers. -> Better handling of non-Y2K-compliant date formats. -> Better handling of extensions.map files with poor line-termination -> Better handling of ovExpandedNodes for filtered case in XML overviews. -> Twister's start.conf file cleaned up. -> Better port binding options for Typhoon and Twister. -> Better counting of newsgroups in active file conversion. -> Better failure messages from activeConvert. -> Better handling of overview corruption in article retrieval. -> Better handling of corrupt compressed articles. -> NNTP chaining is now bandwidth limited. -> Cleanups of auth program interface. -> HTTP Referer field is passed to authentication program. -> HTTP cookie info is passed to post filter. -> num_children for overviews in XML available -> New command-line options documented (see bin/twisterd -help). -> Sample interface template improvements and bug fixes. BUG: Extra newlines in encoded posts removed. BUG: data corruption bug in group deletion fixed BUG: x86 data corruption bug in new active file fixed BUG: fd leak plugged BUG: ovCurrLeftMarkers behaves better when filtered. BUG: activeConvert -downgrade crash bug fixed. BUG: bin/stop now looks for twisterd.pid BUG: Memory leaks fixed BUG: Crash bug on bad command-line arguments in nntpTime BUG: Fixed bug in NEWNEWS handling of cross-posted articles BUG: NNTP LISTGROUP chaining bug fix BUG: HttpDirectory and TemplateDirectory are correctly overridden by the authentication program. BUG: AllowProtocol HTTP now correctly won't match NNTP connections ------------------------- Twister 1.0.2 -> Authentication is now active for HTTP access. -> AuthenticationProgram can set XML variables by emitting a header like "ContextValues: var=value&var2=value" -> Friendly message for web browsers which connect to the NNTP port. -> Support for one-time-use spools with the ArchiveThreshold directive. -> Spool rejection code "0e" for messages which are dropped due to an I/O error or an archive spool filling. -> Support for friendly error pages with the ErrorTemplate directive -> Support for multiple destination hosts/ports in virtual servers with the HttpHostname directive. -> AuthenticationProgram can receive and override HTTP cookies by emitting a header like "HTTPCookie: key=value" -> AuthenticationProgram can redirect on failure by emitting a header like "RedirectURL: http://regserver.company.com/" -> AuthenticationProgram controls HTTP authentication realm with the "AuthenticationRealm" header. -> @post URLs now deprecated; use @article instead -> VirtualServer objects can override look and feel based on Twister URL with the HttpPath directive. -> Better support for "> "-quoting of replied to articles. BUG: URL escaping now correctly escapes percent signs. BUG: AllowReading/AllowPosting control now limits HTTP access. ---------------------------- Copyright (c) 1999-2007 Highwinds Software LLC All Rights Reserved. www.highwinds-software.com software@highwinds-software.com