


	crc32tag 0.25b (freeware) : content plugin for total commander.
	--------



Presentation :
------------
This content plugin calculates CRC32 values, allowing multiple features:
- To check values from SFV files or from tags in filenames. E.G. FileName_[0AA03FF4].bin.
- To rename files to FileName_[CRC32] format.
- To find SFV that are declaring missing files.
- To return current CRC32 calculated from file.
- To calculate CRC32 from MP3 files, ID3 tags regardless (v1 & v2).
- To speed up by skipping already calculated files from cached lists.
- And more.

The 0.25b is a major release.



Installation : 
------------
* First installation advice: A good combination recommended for a Custom Column:

	Caption			Width	Align	FieldContent
	-------			-----	-----	------------
	CRC32 Check		70	->	crc32tag.CheckCrcFrom.Auto_OnDemand_IfBiggerThanSize3
	Type			32	<-	crc32tag.GetCrc32From.Auto_Result
	Declared		44	->	crc32tag.GetCrc32From.Auto_Crc32
	Current			44	->	crc32tag.GetCrc32FromFile.OnlyOnDemand


Usage:
-----
* There is 3 main functions at this time, which are :

GetCrc32From., to get the crc32 value saved previously on its tag or into a related *.sfv.
	Tag_Crc32 : return the crc32 from filename's tag if exist.
	Tag_YesNo : return "yes" or "no", according to filename's tag.
	Sfv_Crc32 : search the crc32 into *.sfv file in the same folder and return the crc32 if match.
	Sfv_YesNo : search the crc32 into *.sfv file in the same folder and return "yes" if match or "no".
	Auto_Crc32 : combination of Tag_Crc32 and Sfv_Crc32 (the priority is given to the tag).
	Auto_Result : return the support's format either "tag" or "sfv"
	Auto_Result_LegacySlower : return the support's format "tag", "tag/sfv" or "sfv". Slower than Auto_Result!



GetCrc32FromFile., to calculate the actual file's crc32.
	NoBrackets : return the crc32 number without brackets (only if tag didn't exist in filename).
	WithBrackets : return the crc32 number withbrackets (only if tag didn't exist in filename, recommended for Multi-Renamer Tool).
	Force_NoBrackets : return the crc32 number without brackets (even if crc32 exist in filename's tag).
	Force_WithBrackets : return the crc32 number with brackets (even if crc32 exist in filename's tag).
	OnlyOnDemand : return the crc32 only when user asking for it (spacebar, or column's selection, ect..).
	OnDemand_IfTagName : return the crc32 directly if crc32 didn't exist in filename's tag.
	OnDemand_IfBiggerThanSize1 : on demand, if filesize is bigger than Size1 (recommended for columns, see further).
	OnlyIfLesserThanSize2 : return crc32 only if filesize is lesser than Size2 (recommended for Overwrite dialog, see further).

CheckCrcFrom., to check if the crc32 value saved previously in tag or sfv matching the actual file's crc32.
	Tag : Get crc32 from tag if exist, possible returns "OK, Wrong !, Read Error !, User Aborted !"
	Tag_OnlyOnDemand : idem, only on demand.
	Sfv : Get crc32 from sfv if exist, possible returns "OK, Wrong !, Read Error !, User Aborted !"
	Sfv_OnlyOnDemand : idem, only on demand.
	Auto : Get crc32 from both tag and sfv (the priority is given to tag), returns idem.
	Auto_OnlyOnDemand : idem, only on demand.
	Auto_OnDemand_IfBiggerThanSize3 : on demand, if filesize is bigger than Size3 (recommended for columns, see further)



Advices:
-------
* To remove an existing crc32's tag into a filename, you can use the Multi-Rename Tool from tc, with the following Regular Expression :
(_| |)\[[ABCDEF0123456789]{8}\]
Note : you can also combine with [=?]Plugin function to remove only the ones who return "tag/sfv" to keep only the sfv value, ect..


* To use NTFS folder\file compression for the cache list. (and in general, any files filled with repetitive characters like log files or some video games files)



Various notes:
-------------
* It is recommended to read 'History Versions' section (scroll below) for changes but also to learn more about this plugin.


* Since 0.25b, the cache list is greatly speed optimized and size increased by using 256 sorted lists of 256 lines each:
The number of 256 lines per list was determined as the best compromise between speed access, total storage, memory usage, average before overwriting, ect...
You will find the other values for compairing in detail into the crc32tag.cpp source file.
You will find the differents compilated versions for 32, 64, 128, 256, 512, 1024 and 2048 at my patreon page if needed.

Here is only the one that is used in the compilated plugin, which is 256 (MAX_CACHE_LIST_SIZE):
--------------------------------------------------------------------------------------------
- Accessing cache list speeds theorical compare showing the advantage on the new 256 lists x 256 lines versus the old version using a single list of 8001 lines:
  (8001/256)/2 = average of 15.62x faster, for a 65536/8001 = 8.19x bigger cache list size.

- Accessing cache list speeds theorical compare showing the advantage on the new 256 lists x 256 lines versus the old single list if it had to keep matching the total sizes:
  64008 / ( 65536 /  256) = 1 to 250x or 100% to 25000%. Average: 125x faster.

- Refreshing before starting overwriting theorical:
  New cache lists of 256x256(65536) total lines per 3150 files refreshing\adding : 256/(3150/256) = 20.8x refreshes in average before overwriting.

- Space taken in memory and disk for 256 cache lists (non-theorical except some of the cache file size):
  256 = 65536 cache lines, +65MB to the file manager (x86) in memory, should give a cache file size of +-24MB once filled at 100% with datas.


* Since 0.16b, the CheckCrcFrom for a file with the *.sfv extention will be treated alternatively, and will return a quick test of the sfv : e.g. "3/10 missing !", "OK" (when OK means here that all the listed files in the sfv are found, NOT that their listed crc32 are correct!) or "Empty !"

* Since 0.14b and thanks to the cache function, you can use all columns combinations that you want, the crc32 from a file will be calculated only once, and the next request will be returned with the value contained in the cache.

* About the cache : cache is rotating and can containing 65536 values as default compilated since v0.25b.
It permit to fix the problem when using the Multi-Renamer tool (values are constantly re-requested when working on it), to resume or remember large folder results, ect...
The cache match a value only when : fullpathname & modified time & filesize are equal to the actual file status.
If at least one of these three value does not match, the cache value from list is ignored and the crc32 is calculated again from the file.
Be warned : some editors are not saving the "modified time value" when modifiying a file (as example hexworkshop or ResHacker).
That means the crc32 returned from cache may be wrong in this case (also true in case of hd failure, virus, ect..).
To remedy and ensure the result, you can refresh the cache by pressing the F2 or CTRL+R (only since TC7, for previous versions, you'll have to close TC to clear the cache!)
Refreshing the cache : it is clearing all crc32 values from the actual working folder and its subfolders (Optional with Int_CacheRefreshAlsoSubReps).
So, by pressing F2 from c:\*.* for instance, all cached value from c:\*.*, but also from c:\temp\*.* and c:\windows\*.* will be cleared.
To make it clear, if you want to clear the cache list entries for c:\temp\*.* and not for c:\windows\*.*, press F2 only when you are inside the c:\temp\*.* folder.
At this time (0.25b), the cache is rotating but the refresh function does not reorganize values yet (it should be optimized in future).


* Since 0.17b, the cache can be saved into a file and reloaded (disabled by default), using the 3 ini entries : Int_CacheSaveToFile=0, Int_CacheLoadFromFile=0 and Str_CacheFileName=crc32tag.cache.txt.

About Str_CacheFileName, if you do not specify a complete path, the %commander_path% will be added, so as instance, Str_CacheFileName=LOG\CACHE_CRC32TAG.LOG will be interpreted as : %commander_path% + \ + LOG\CACHE_CRC32TAG.LOG
If the Str_CacheFileName is empty, the values Int_CacheSaveToFile and Int_CacheLoadFromFile will be set to 0 internally whatever their initial values.
Since TCs older versions than tc7 does not have a refresh function for content plugins, I do not recommend to use the cache file functions for these.


* To find a crc32 into a .sfv, the plugin is searching in order :
"PathFileName.sfv" (from PathFileName.sfv), if not found -> "Path*.sfv" (all .sfv in folder, excepted the previous PathFileName.sfv)
Note : after some tests it seem that having a lot a sfv files (+100) in the folder may reduce read performances (edit:fixed since 0.14b).


* Since 0.14b, 3 value have been added (editable in BYTES ONLY into contplug.ini since 0.17b):
Size1 = 1Mb, Size2 = 500Kb, Size3 = 250Mb (Size3 = 50Mb by default since 0.25b)
They may useful to set functions optional : 
As example, on overwrite dialog, Size2 can be used to directly calculate crc32 of little files(filesize<500Kb), which will permit to avoid to have to press the [More Options]->[Compare] to know if the files are equals in content.
If the file is bigger than 500Kb, the crc32 is not calculated automatically to avoid delays.


* Actually (0.13b, edit:fixed since 0.14b, and optimized since 0.25b), you should not use this plugin with Multi-Rename Tool from tc to create crc32 tags : cache is needed to avoid troubles with big files.
See http://www.ghisler.ch/board/viewtopic.php?t=12326 for more infos and below in todo list.


* (until 0.22b) This plugin use the source CRC32.h from Brian Friesen (brian_friesen@yahoo.com),
which has been modified for plugin's needs.
Licence from CRC32.h :
--------------------------------------------------------------------
Copyright (c) 2006, Vilppu Tuominen, University of Tampere
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the University of Tampere nor the names of its
      contributors may be used to endorse or promote products derived from
      this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS "AS IS" AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE REGENTS AND CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
--------------------------------------------------------------------


* (since 0.23b)
 crc32.c -- compute the CRC-32 of a data stream
 Copyright (C) 1995-2006, 2010, 2011, 2012 Mark Adler
 For conditions of distribution and use, see copyright notice in zlib.h

 Thanks to Rodney Brown <rbrown64@csc.com.au> for his contribution of faster
 CRC methods: exclusive-oring 32 bits of data at a time, and pre-computing
 tables for updating the shift register in one step with three exclusive-ors
 instead of four steps with four exclusive-ors.  This results in about a
 factor of two increase in speed on a Power PC G4 (PPC7455) using gcc -O3.


* This plugin is compatible and can be used in addition with TagHOT (freeware) from (edit:dead link) http://www.cs.uta.fi/~vt72556/software/taghot/ 


* For support, questions, feedbacks, ect.., you can go to this topic : http://www.ghisler.ch/board/viewtopic.php?t=12341


* DEU language from crc32tag.lng is a maintained contribution from TychoBarfy, thank you!



History Versions :
----------------
* 0.25b, (MAJOR RELEASE)
- NOTE: Do not forget to enable both ini entries: Int_CacheSaveToFile=1 and Int_CacheLoadFromFile=1 to benefit all of the cache list updated functions.
- Speed optimisation: Added ini entry: Int_MaxBufferSize, which should greatly increases the reading speed in some cases (the previous versions were fixed 4096B buffer)
- Speed optimisation: Now GetCrc32From.* functions will return empty field for SFV filetype when ini entry Int_TreatSfvAsSpecial is equal to 1. This is to avoid searching for SFV files for another SFV.
- Speed compromise: Now GetCrc32From.Auto_Result will only return "tag" instead of "tag/sfv" (skip SFV searching if file is already covered by tag in name).
- Added GetCrc32From.Auto_Result_LegacySlower which has the same functions as the previous version of GetCrc32From.Auto_Result (more precise).
- Added GetCrc32From.Auto_Result_LegacySlower string entry to crc32tag.lng.
- Str_Delayed default ini entry is now "- - -" against "- - - - -" before.
- Speed optimisation: Greatly optimized cache list reading speed in memory by using 256 sorted lists of 256 lines each.
- Speed optimisation: Now accessing cache list in memory should feel almost instant: 8001\256 = 31.25x faster than before!
- Longevity Optimisation: Now the life of cache list entries before being overwrited should be way much longer, thanks to both the 256 lists sorting and the increased total size: 65536 entries vs 8001 for older versions.
- Longevity Optimisation: Because of the new bigger cache list entries count (16+65536 vs 8001), the memory usage increased from +3MB to +65MB (tested on x86 version).
- Safety: basic check at loading the cache file to automatically ignore and replace in case of older/corrupted/incompatible content.
- PrivateInt_CacheListIndex ini entry is now obsolete and will not anymore be loaded nor saved to ini file.
- Default now ini entry Int_Size3 is equal to 52428800(50MB) (250MB on previous version)
- Changed returning string "corrupted META?" to "BAD META?"
- Speed optimisation: More accurate searching for the matching SFV file. Searching order until SFV is found is now: E.G. c:\folder\file.txt --> file.sfv, file.txt.sfv, folder.sfv, *.sfv or f*.sfv, sfv not found.
- Speed compromise: Added ini entry Int_FolderSearchOnlyMatchingFirstChar_FASTER. If set to 1, search only for files matching the first letter. E.G.: c:\test.txt --> c:\t*.sfv. Otherwise: c:\*.sfv
- Speed compromise: Added ini entry Int_FolderSearchMaxFilesCount. -1=search for all (can be very slow if LOT of sfv in current folder) 0 or more=Maximum searching before returning CRC not found in SFV.
- Now compiled with VS2012EX

* 0.24b,
- use crc32tag.ini from plugin directory if it already exists otherwise use contplug.ini as before (tbeu)

* 0.23b,
- Corrected spanish words in crc32tag.lng (Sombra)
- added 64 bit support (tbeu)
- full support of Unicode (tbeu)
- crc32 calculation is based on zlib 1.2.7 avoiding inline assembler (tbeu)
- Str_CacheFileName ini entry (from contplug.ini) does not support relative path anymore

* 0.22b,
- Fixed a bug where the WithoutMETADATA value was not returned correctly (showing sometimes a wrong crc32)

* 0.21b,
- Added WithoutMETADATA & OnDemand_WithoutMETADATA (currently working only for ID3 from MP3 and WAV files).
- Now it permit to calculate crc32 from MP3 files, ID3 tags regardless (v1 & v2).
- The result of WithoutMETADATA is also saved into cache list; like "c:\file.mp3, 11, 2001/1/01, 01:01:01NOMETAFFFFFFFF"

* 0.20b,
-Now GetCrc32FromFile automatically check the cache value before returning a delaying or the "(on demand)" value.
-Now CheckCrcFrom return a non-delayed value for the Tag based entries.
-Now CheckCrcFrom may check the cache value and automatically delaying the result instead of "(on demand)" value.

* 0.19b,
-Now ALL internal string comparison functions are no more case sensitive.
-Added ini entry : Int_CacheRefreshAlsoSubReps (0 = don't refresh subfolders in cache)
-Fixed a problem where an identical value could be saved in cache more than once when Int_UseCache=0
-More messy code than ever!

* 0.18b,
-Fixed the cache file that was not unleashed properly after plugin loading.
-Added the ini entry : Int_UseCache=1 (0 = never read crc on cache but still writing on it).
-Added the private ini entry : PrivateInt_CacheListIndex for internal needs.
-Cache is now saved as it is (with its empty cleared lines).
-Some minors fixes and code cleared.

* 0.17b,
-Added DEU language for crc32tag.lng (contribution from TychoBarfy, thank you!).
-Added pluginst.inf for automatic installation (contribution from TychoBarfy).
-Now some values can be edited directly into TC's content plugins ini file : contplug.ini.
-Now Special treat for *.sfv file is optional but enabled by default (see into ini file, value : Int_TreatSfvAsSpecial).
-Increased cachelist from 4001 to 8001
-cleared some temporary code, which decreasing plugin size from 112kb to 70kb
-Fixed a bug where sfv content was incorrectly interpreted with line bigger than 512, the new value is set to 2048
-Cache can be saved and reloaded from a cache file (disabled by default into ini file)

* 0.16b,
-Now CheckCrcFrom for a file with *.sfv extention will be treated alternatively, and will return special related values.
-Added return string "x/x missing !", "Empty !" and "OK" for new sfv CheckCrcFrom special function.
-Now string comparison is no more case sensitive for a sfv content : PROUT.BIN listed in a sfv will match the filename PRoUT.BiN

* 0.15b,
-Fixed a "bug" where some functions did not worked well with path bigger than 127 chars.

* 0.14b,
-Added a rotating cache, which improve a lot of things (list is too long for here)
-Improved speed when scanning a folder with multiple sfv
-Added a refresh cache function by folder/subfolders. (tc7 only)
-Added 3 variable Size1 to Size3 (should be editable in future)
-Added OnDemand_IfBiggerThanSize1, OnlyIfLesserThanSize2 for GetCrc32FromFile
-Added Auto_OnDemand_IfBiggerThanSize3 for CheckCrc32From
-Added tc's language support for french and spanish
-plugin is now compressed by upx 2.03
-Minors fixes, major release :)

* 0.13b,
-Added .sfv support (read-only) for GetCrc32From and CheckCrc32From.
-Added functions Sfv_Crc32, Sfv_YesNo, Auto_Crc32, Auto_Result, for GetCrc32From.
-Modified OnDemandIfTagName to OnDemand_IfTagName for GetCrc32FromFile.
-Added functions Sfv, Sfv_OnlyOnDemand, Auto, Auto_OnlyOnDemand for CheckCrc32From.
-Modified some messages to match better with tc (crc ok->OK, ! WRONG->Wrong Crc! ect...).

* 0.12a,
-Added the message "! aborted" and fixed a misinterpretation when user was canceling crc32 calculation.
-Added the message "(on demand)" to signal when a value need to be asked.
-Added the message "........" to signal when a crc32 result is incoming (delayed).
-Some minors fixes and code cleared.

* 0.10a,
-First alpha release, for testing purpose preferentially.



Todo list
---------

* (minor) log and stats functions (last check time, last result, ect..) [partially done with cache function]

* thread safety (tbeu)

* dynamic cache list (tbeu)

* (minor) To support SFV UTF-8N (UTF-8 without a BOM header)

* (minor) To support SFV non-UTF-8 containing rare characters

* new field content: cached date



Supporting my work:
------------------

This is a freeware, but feel free to support me for few bucks, at my patreon webpage: https://www.patreon.com/byblo

You can alternatively sending me some currency at my paypal account: byblo@hotmail.com.

That should encourage me to keep working on my tools and plugins, to eat more hamburgers, but also to buy some english grammar books.

You can also write some feedback about this plugin here http://www.ghisler.ch/board/viewtopic.php?t=12341.



Enjoy :)


	2006-2022 byblo(was here)



