See for lines with "CHANGE" to see what you MUST do when upgrading from a previous version. v0.6.0 ------- - Made some changes to the download logic, it should now pass more gracefully on errors. - Extra options for suck can be now specified like other cfg values by bowser or pcfg - Fixed suckkillfile support - again. :) suckkillfile can be specified now by bowser or pcfg. - Increased the maximum filename length to 255 - Files can now be tagged with keywords using command 'pkw' - Script 'pv_key keyword1 keyword2 ...' can be used to view the files that match ALL the given keys - Added a new command 'pleechwww' to download image galleries from the web. It works by taking as input an url, keywords to match (as regexp) and a destination area. The page the url points to is assumed to contain links to image galleries, and that the link description includes the keyword if its to be leeched. All the words in the link desc ('bob sent images of these lovely bananas') will be added as keywords for the received files in the database (i.e. you can do e.g. 'pv_key bob bananas' to see the files). Browsed links will be marked up in the database and won't be rebrowsed on further runs of pleechwww. For WWW leeching, the standard pimppa md5sum checks etc apply. The fetched files will be renamed according to the link description (since the web-gallery images are usually named just "01.jpg" etc ...). WWW leech requires wget to be installed, and is not supported by the GUI bowser. - Misc small code changes and cleanups CHANGE: bash> mysql -u root -p < sql/058to060.sql v0.5.8 ------ - Made build compatible with mysql 5.0 series - Fixed a bug where the article subjects werent properly inserted as descriptions using combination of -i and -s switches for pleech. - Fixed a glitch where md5sum collision stopped the inclusion of the file only if pleech was run in verbose mode. - Added a new way to verify incoming files, these are called 'prune scripts', which are filetype specific. These resemble the commands for testing incoming files, with the difference that if a prune-script returns a non-zero value, the file is immediately deleted. This allows the scripting of arbitrary complex, possibly file-content based spam prevention schemes that the file should pass before its accepted by PIMPPA. More info in README. - Added a prune script "p_prunejpeg" to delete too small jpegs. It must be manually activated. See README. - Added a script "p_contexts" to print out the current contexts - Some minor code and doc cleanup CHANGE: bash> mysql -u root -p < sql/057to058.sql v0.5.7 ------ - More options for suck can now be appended in "src/pimppa.h". - Removed "ydec" dependency, as current uudeview supports yyencode - Added a table defining the existing contexts + some bowser combo boxes. - Added keyword hunting behaviour (accept only messages that contain specified keyword pattern in the subject line), pleech switch '-k'. See README. - Tweaked autoconf stuff - src/news.c didn't create directories properly if the last path component wasn't '/'. Fixed. - (Group in "~/.my.cnf" is now by default [pimppa], not [client]) This change was not good!!! Backtracked. - Tried to fix mysql_init() related crashes seen on some MySQL-dists [but it appeared to be a bug in mysql v3.23.49]. - Tried to fix build for FreeBSD - When parsing filename from the subject: line, the parsing is now stopped at " -character (yenc stuff is often in "file.jpg" style format on the subject line) CHANGE: bash> mysql -u root -p < sql/056to057.sql v0.5.6 ------ - Added yencode (yEnc) decode support using "ydec". (http://sf.net/projects/yenc/) - Added "-d" and "-v" switches to "pf" to find and print file descriptions. - Moving files to 0DOOM now marks them as offline. - Tried to fix the configure script for various mysql installations (took some code from gmysql, my compliments) - Added script "p_leechctx" to download all such newsgroups whose target area belongs to a given context. - Added script "p_rename" to rename files in database and on the disk. Thanks to A.E. for the script. (Note that the current script works badly if there's several files with the same name in the database). - Added scripts "pvm_name" and "playem.pl" to display movie files. They require "mplayer" for usual file types and "realplay" for rm. Scripts were submitted by boz boz. Thanks. - Small GUI changes v0.5.5a ------- - create_database.sql was broken - ptrans could fail if the user didn't have write access to the dir where it was executed from. Fixed. v0.5.5 ------ - Enhanced security, especially important for multiuser systems. Accessing pimppa DB now has a password, but you don't have to type it all the time, as it is automatically read from a file. CHANGE: Edit ~/.my.cnf to contain ---cut--- [client] user=pimppa password=your_new_pw ---cut--- and make sure other people can't read the file, e.g. $ chmod 0600 ~/.my.cnf Change mysql password for the user: $ mysql -u root -p --execute="USE mysql; UPDATE user \ SET password=PASSWORD('your_new_pw') WHERE user='pimppa';" If you already have some mysql user other than that, you probably know what you're doing; just give that user priviledges to the pimppa database. - More intelligent (& conservative) free space check; pimppa doesn't now download more messages than the tmpdir has space for tmpfile and decoding. - Filenames from article subjects in format like "C:\bull\bull10.jpg" will now be correctly extracted without the path. - Entering context in pnewarea was broken, fixed - Changed Bowser add_newsgroup default server id to 1 - Changed the Bowser selection mode to one where you can select multiple files by pressing down ctrl, and select blocks of files by pressing down shift. Thanks to Boz Boz for idea. - Added "-n" to suck header leech phase. Seems to speed things up. - Viewing scripts now read the file viewer from p_misc key CFG_VIEWER and default to gqview if not found. (Thanks to I.O.) v0.5.4 ------ - Added ability to specify RegExp -rules to classify files based on their file names. (Like "Always move goat*.jpg to filearea 2goats") - Context support. Fileareas can now be grouped to contexts, so that assign patterns and classifying rules function only on areas belonging to the same (or global) context. - Renamed area *DOOM* to 0DOOM - Restored support for user-specifiable "suckkillfile". - Added some more explanation about dl behaviour to the README CHANGE: Upgrade to suck 4.3.0. bash> mysql -u root -p < sql/053to054.sql v0.5.3 ----- - Support for multiple newsservers - "pleech" can now insert message subjects as file descriptions with switch -i. "bowser" does this automatically. Note that the used method can't handle all cases correctly. - Fixed a couple of odd-situation crashes in downloading news. Thanks to Sebastien for patch. - Messages with no part info were not downloaded in 0.5.2. Fixed. - Changed datatype of p_groups/g_last to signed integer to allow specifying max amount of dl'ed messages to suck with a negative value. - Fixed crash if we get Message-ID: header line from the server in unexpected case (like "Message-Id:" for example). - Modifications to passign - Added sloppy download mode ("-s") to pleech. It will accept, decode and adopt everything, without doing any checking anywhere. CHANGE: bash> mysql -u root -p < sql/052to053.sql v0.5.2 ------ - Removed P_DATADIR. User selectable tempdir is used instead. - GUI operations now output to a global info window which can be closed and re-opened. - Changed assign pattern scheme to mark last letters in filename as indexes only if there's two or less of them. - Target area of any assign pattern can now be set to 0, which means that the particular pattern is disabled and it won't be replaced if matching files are moved or adopted. Responsibility of sorting the matching files is left to the user. - Added areaflag AREA_NOASSIGN. Files will never be assigned to such areas, nor patterns will be created for those areas. - Moving files in GUI didn't update assign patterns. Fixed. - Added script "p_gtog" to toggle the enable/disable status of a given newsgroup pattern. Thanks to Newsboy for the script. - Modified script "p_groups" to be able to also print the disabled groups. - Added some cheap shortcuts to the GUI for padopt, ptest, ptrans, pmarkoff, passign and pmd5sum. - Relaxed the default configuration parameters to allow more files. Old strict rules confused people. - Added simple configuration tool "pcfg". Without arguments, it prints the current setup from table p_misc. You can use it to insert/replace cfg keys like "CFG_FILETYPES" without recompiling. You can also modify them by "bowser". - Most configuration values are now primarily taken from table p_misc instead of "pimppa.h", enabling more dynamic configuration. - Theoretically you can now run multiple "pleech"es at the same time, perhaps for different nntp-servers. It probably won't work (atleast not well), as there's no database-level concurrency control. - Restart-option "-r" for "pleech": read all messages from newsgroups. - Two-phase news downloading: first, download all headers. Then check for suitable, complete postings. Download those and skip the others. At this point the code is kludgy and messy and likely to cause trouble. - "pleech" now prints BPS and decreasing article count if option -q is not used. - Added pimppa database uninstall script to "extras/". - Fixed the stopping problem of pchkfn - Amount of numbers & non-numbers required in the incoming filenames can now be specified (as filenames like "a1.jpg" tend to annoy people or be rubbish). - Dots were not escaped in the regular expressions of allowed filetypes in "pimppa.h". Fixed. - Now all pimppa utils commit suicide if there is any other problem with the mysql database queries than a parse error in the query statement. This prevents further disasters in case the database is broken. CHANGE: After install, run shell> passign v0.5.1 ------ - "pleech" has never been accepting the given username & password. Urk, what a mistake. Thanks to Hamish for report. Fixed now. - The default amount of time pimppa waits between articles while leeching news can now be changed from "pimppa.h". - "pmv3" reported the amount of moved files wrong. Hopefully fixed. - Added lenient article selection option "-l" to leeching news in pleech. It will download every article regardless of subject/content. File selection is done after decoding. (Useful e.g. with some japanese newsgroups where they don't write filenames to the subject in western characters) - FTP leeching is now part of "pleech". Script "ftpleech" removed. Note: I haven't tested it much, let me know if it doesn't work! Or give me account to a site with interesting content so I can test it better. ;) - Fixed some leech crashes when encountered filenames were too long for the system. Such files are now coldly skipped. :( The current max filename length is 128 chars (more requires a database change in addition to "pimppa.h" modification). - Joined all similar file adoption code to "adopt.c" If something broke, let me know. - "padopt -m" now needs a default "move to" area argument. Warning: that also kills all files from the current dir not liked by pimppa - Added verbosity to news leeching if -v is used - Fixed bug in dupechecking of filenames with no dot in filename - Reduced the amount of unnecessary downloading while leeching news. From now on, suck passes a hook program "pchkfn" headers of each news article to decide if it should be downloaded. "pchkfn" tries to determine the filename from the Subject: line and check if it would be accepted to the system. This will make the newsleech more "picky" (some good stuff might be skipped), but the amount of unnecessary downloading of duplicates and crap is much reduced. See "src/pimppa.h" to change the accepted filetypes. Fixed files "alt.keep" and "suckkillfile" are no longer used. Thanks to Newsboy for the script & idea. - Ability to leech files named like *.r00, *.r01 and so on, without having to state each extension explicitly (uses regexp). v0.5.0a ------- - Fixed suck call in news leeching to be able to read the created suck history file, now "suck.db", as some suck versions don't handle the "-HF" parameter properly. Thanks to Newsboy for this and the next one. - Adding new fileareas with "pnewarea" was broken v0.5.0 ------ - The damn "suckkillfile" didn't stay fixed. I don't get it! - "passign" gives now a little statistics of how effective the pattern scheme is - Kludged "ftpleech" to delete the empty directories fget creates (the simpler the applied patch, the better, I think) - "ftpleech" now takes a special URLFILE as argument (see README). - Changed "pmv" file overwrite policy to overwrite only broken files - Changed "prm" negative assign creation routine not to replace existing assigns unless they point to the area we're deleting the files from, or if forced - Fixed "padopt" not to create assign patterns for incoming areas - Fixed a foreign key database anomaly with -1 areas. Now a specific filearea called *DOOM* exists with area_id=-1. Also the area ids are now smaller to save a little space. If someone wants over 30000 fileareas on her system, let me know. - Added MD5 checksum support for files in system. Thanks to W for suggestion. This makes duplicate discarding MUCH more effective. - Added command "pmd5sum" to calculate/print/lookup/scan MD5 checksums. Also padopt/ptrans/leech consider MD5 checksums. Default operation is strict (delete colliding files except the oldest one). See "src/pimppa.h" to change this. - Added "pclean" option "-t" to remove files from database which have ended up on area -1. You can also use area "-1" to move spam files to (they'll be deleted, but their md5sums kept). In that case, do not use "pclean -t". CHANGE: bash> mysql -u root -p mysql> USE pimppa; mysql> ALTER TABLE p_areas MODIFY COLUMN area_id smallint(5) DEFAULT '0' NOT NULL auto_increment; mysql> ALTER TABLE p_assign MODIFY COLUMN a_dest smallint(5) DEFAULT '0' NOT NULL; mysql> ALTER TABLE p_files MODIFY COLUMN file_area smallint(5) DEFAULT '0' NOT NULL; mysql> ALTER TABLE p_groups MODIFY COLUMN g_dest smallint(5) DEFAULT '0' NOT NULL; mysql> INSERT INTO p_areas (area_id, area_name, area_path,area_targets) VALUES (-1,'*DOOM*','/dev/null', 'End of the Line'); mysql> ALTER TABLE p_files ADD COLUMN (file_md5sum char(16) BINARY DEFAULT '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0' NOT NULL); mysql> ALTER TABLE p_files ADD INDEX (file_md5sum); bash> pmd5sum -v -s (all this will take some time) v0.4.6a ------- - I had managed to break the file moving routine in news leeching. Fixed. - Fixed the RPM .spec file v0.4.6 ------ - Preliminary FTP leech support using "(lib)fget". See "patches/fget_patch.txt" for information. The patched version is used by "scripts/ftpleech", which recursively leechs an URL while letting pimppa select which files to skip. The downloaded files are automatically moved to the fileareas. - Added "-a" option to pbackup to include certain areas - Fixed needletter/neednumber bug in pclean - New command "pchkfn". It checks if a file would pass pimppa rules and duplicate checking. Returns "0" if yes. It can be used by your own download scripts. - "padopt" now converts filenames to lowercase if the rule is on - Some code cleanup v0.4.5a ------- A small bugfix release. - Fixed "sql/create_database.sql" to use correct table and flush priviledges (patch supplied by anonymous). - Added optional NNTP username and password options for "pleech" and "bowser" (in bowser, see settings/prefs/misc). - Renamed "pkill" to "prm", because pkill is part of procps package. I guess they wouldn't rename if I asked. v0.4.5 ------ - Added a FAQ - Added some new gadgets to the main screen of "bowser". - Added manual refresh (w/ right button) to preferences in "bowser". - Added "kill file" option to "bowser". Its same as "pkill". - Added a right-button pulldown menu to SQL Where -box of "bowser". - Added "all areas" option to "bowser" arealist. Naturally slow on huge databases. - Added selective newsgroup leech to "bowser". - Added ".avi", ".mng" and ".mp3" to default "alt.keep" - If there's "PIMPPA_NNTPSERVER" key defined in "p_misc", the data field is used as newsserver by "pleech", instead of P_DEFAULT_NNTPSERVER ("bowser" already uses that), unless a server is specified on command line. - Area dupecheck RegExp can now specified to "pnewarea" with "-T". - Fixed bug in news.c assign, now assigns only if the destination points to an area defined by area_targets of the default area. - More intelligent free-row-data routine in "bowser", increased speed - "bowser" now defaults to "view all files since 1 days" behaviour on startup. v0.4.4 ------ - Various modifications, adds & fixes to the GUI. Now pimppa is *almost* usable from the GUI. ;) ;) - Added "scripts/pngcrush.stub" script. You can use it (with pngcrush) to optimize (transform) PNG images. - Added relaxed regexp style dupechecking (see README/9.3). - Fixed(?) transformed dupecheck related bug in news.c - CHANGE: mysql> USE pimppa; mysql> ALTER TABLE p_areas ADD COLUMN area_targets TEXT NOT NULL; mysql> UPDATE p_areas SET area_targets="$"; - Updated documentation - Added "pimppa.spec" for RPM building v0.4.3 ------ - "pclean -f" now really deletes the failed files from the database and doesn't just mark them offline. Note that backuped failed files will not be deleted. - suck's history file is now stored in P_DATADIR. v0.4.2 ------ - New command "pdesc" to describe files - Added trivial script "p_groups" to show current newsgroups which are not marked as disabled. - "pbackup" now produces such filelists that they can be easily reloaded to SQL database later if needed. Con: they're not as human readable as previously. - Newsgroup downloading from "bowser" fixed, I hope v0.4.1 ------ - There was a nasty bug in "bowser". It always downloaded news in "nazi" mode, discarding all files with no assign patterns. - Code cleanup, Bowser has been further gnomizied. v0.4.0 ------ - PIMPPA build now uses "configure", so the installation has changed. In future, do "./configure ; make install" to install pimppa to subdirs of "/usr/local/". You should remove old PIMPPA executables and scripts by hand. - "Bowser" has been almost completely rewritten for GNOME. - "Bowser" now uses gqview to view pictures. Still no internal viewer. Blah. :( - Added "-n" option to "pkill" for killing nicely. ;) That is, the file is just marked offline, no assign patterns are changed, and the actual file is deleted. v0.3.3 ------ - "pleech" no longer needs NNTP host given as argument. If its not given explicitly, pleech uses one defined in "src/pimppa.h". - Some assign patterns were incorrectly escaped, fixed. - New option "-a " for "padopt". It can be used to make padopt visit only certain areas defined by the given SQL area_name pattern. Example: "padopt -a %slimy%" = all areas with names matching 'slimy'. - Fixed incorrect string length calc bug in "padddir". - Tried to fix some bugs concerning filenames with spaces, '-characters, etc. - Added new script "p_con". It prints out the contents for a given backup volume id in 'Area | Megs' format. - Added new script "p_loc". Its given an area name (or pattern) as argument and it prints out how much different backup id's contain stuff from that area. - Added "-o" option to "pclean". It deletes files marked with FLAG_OFFLINE which have a backup id of 0 (==not backupped). - "padopt -m" now skips existing (unbroken) destination files - "ptest" now considers numeric exit values of filetype test commands if 'type_testokstr' field in "p_type"-table is empty. In that case, the exit value 0 should mean the file is ok, otherwise its considered broken. - The "patches/" directory contains a file to patch mpeg_stat v2.2b in such a way that it gives PIMPPA-compatible exit values (0==file OK, 1==error) - Added script "pv_last" which views files that have arrived since the last execution of the script. v0.3.2 ------ - Renamed some utils in "bin/" so that all the names are consistent. * adddir => padddir * adopter => padopt * assign => passign * automaton => pleech * cleaner => pclean * fs => pf * mark_offline => pmarkoff * pback => pbackup CHANGE: If you have some scripts (crond?) referring to those utils, modify them accordingly. - Fixed 'incorrect starting area' bug in "pbackup". - Added 'leech news' option to "bowser". - Added 'move' (-m) switch to "padopt". - Added 'purify' (-p) switch to "pclean". - Added 'simulate' (-s) switch to "pclean". - Added 'slaughter' (-z) switch to "pclean". - "pclean" can now perform multiple actions at one run. Their order is still predefined, though. - Fixed a bug in "bowser" which caused it to crash if too many files were tried to be viewed. - Fixed "pclean -a" operation. Should work now. Or maybe not. - There was a date related bug in "pv_since". Fixed. v0.3.1 ------ - Fixed bug in "automaton" which caused it to quit if some newsgroup had nothing to decode in it. - Fixed bug in internal p_cp() routine which caused it to fail if the destination wasn't a directory. - Forgot to add "USE pimppa;" line to "sql/create_tables.sql". v0.3.0 ------ - Now developed using MySQL v3.22.25 RPM version. Source files look for mysql.h include from the location assigned by the RPM. I'm too lazy to learn how to write a 'configure' script. - Added "NEEDLETTER" option to "pimppa.h". Filenames with no alphabetical character before the last '.' are as default discarded by "automaton". - Added "Move" option to "bowser". It moves files around. - Added "Select All" and "Unselect all" menu choices to "bowser". Filemenu options (Delete, View, Edit, Move) now affect all the currently selected files. - "pmv" now moves files from current dir to any filearea. Filearea related to the current directory is found out by using getcwd() and the area database. - "pmv" now mercilessly overwrites any existing destination files. - Removed obsolete "pmv2". - Added optional nazi behaviour for "automaton". Switch "-n" makes automaton discard all downloaded files which do not match any already existing pattern in 'p_assign' table. - Added optional nazi behaviour for "ptest". Switch "-n" makes ptest delete all files which fail the integrity check. Use with CAUTION. Incorrect integrity check tools or invalid values supplied to p_types -table may cause massive file loss. - Added transformed filename recognition to "automaton" dupechecking. - Misc small changes to "automaton". - "cleaner" is now able to delete all files which failed integrity check. - Some further modifications and fixes to "cleaner". - Minimun filename length of which an assign pattern will be created (by various pimppa utils) can now be specified in "src/pimppa.h". Default is 4 (filename extension is not counted). - PIMPPA's MySQL username changed to a more logical one. Access features are now a little better too. User in "etc/passwd" is not needed. CHANGE: 1) Delete user "bbs" from "/etc/passwd". 2) Enter directory "pimppa/sql", type shell> mysql -u root -p mysql -u root -p mysql> use mysql; mysql> delete from user where user='bbs'; mysql> delete from db where user='bbs'; mysql> use pimppa; mysql> alter table p_assign change a_dest a_dest mediumint(8) DEFAULT '0' NOT NULL; mysql> quit shell> mysqladmin reload - Negative destination area in assign patterns now causes matching file deletion by automaton. Necessary db change was integrated to above. - Assign pattern scheme changed again into a better one (hopefully). CHANGE: Recreate your assign patterns using "bin/assign -c" - Hopefully fixed filename-mangling bug in "ptrans". - Modified "automaton" to read newsgroup names from new pimppa table called "p_groups". The table also contains default destination areas, e.g. all decoded stuff from the news will be moved to that destination area, unless some assign pattern matches. CHANGE: - Create the new table shell> mysql -u root -p mysql> use pimppa; mysql> CREATE TABLE p_groups ( g_name varchar(255) DEFAULT '' NOT NULL, g_last int(10) unsigned DEFAULT '0' NOT NULL, g_flags mediumint(9) DEFAULT '0' NOT NULL, g_dest mediumint(9) DEFAULT '0' NOT NULL, PRIMARY KEY (g_name) ); mysql> quit shell> mysqladmin reload - Use "rc2sql" to convert your existing .rc files to the p_groups -table. The groups will be mapped to area 0, so use "bin/pnewarea -I 0" to create an actual destination area for them. - New area type: AREA_INCOMING. Imaginary areas are no longer necessary. Contents of incoming areas will not be backupped or transformed. You should assign newsgroup destinations to incoming areas. - You can now specify an SQL-style newsgroup match pattern for "automaton" with argument [-g ]. Default is all. - Added newsgroup add/remove tool to "bowser". - PIMPPA utils now perform sql string escaping, so that filenames with characters like '\' and "'" are accepted to the system. - Delay times of "automaton" can now be specified on the command line - Changed the use of external shell "mv" and "cp" commands to internal ones. The internals are not as efficient, but it's cleaner than using a shell command, and they won't collapse when encountering weird characters in filenames. - Several minor modifications to "bowser". - New command "bin/pkill". Deletes files physically and from the DB, giving them a negative assign pattern so that they will be next time 'DELETED on sight' by "automaton". Use with CAUTION. - Added several options to "pnewarea". Now you can specify area to be AREA_NOTRANS, AREA_INCOMING and area_id upon creation. - New command "bin/pnewgrp". Its used to add new newsgroups to the system. v0.2.0 ------ - A GUI called "bowser". See "gui/" and "bin/bowser". GUI compilation requires GTK+ v1.2 or newer. The GUI is still mostly a toy, but you might get some overview from it. - Better "pattern matching" scheme. Should cover some more cases and operate in a more orderly fashion. :) CHANGE: Old patterns do not function anymore. Use "bin/assign -c" to create working ones. - Fixed a trivial path bug in "scripts/p_maint" - Added a numbercheck option to "cleaner". - Incorporated the functionality of "pmassdel" and "cleandir" into "cleaner". Obsolete two utils have been deleted. v0.1.1 ------ - Fixed a bug in file moving of "automaton" - Fixed a bug in comp.os.linux.accounce message. ;) - Added my public PGP key to the README file v0.1.0 ------ Initial release version