I need to replicate the behavior of MySQL utf8_general_ci
collation in PHP. Strictly speaking I need to detect what whould be considered different and what would be considered the same. The case independent part is easy. The problem is utf_general_ci
considers characters with diacritics and characters without diacritics to be equal: e = è = é etc.. To replicate that comparison, I'd need to have a way to replace è -> e, é -> e.
The method that comes to my mind is:
echo iconv("utf-8", "ascii//TRANSLIT", "é");
One problem is iconv
behaves differently depending on current locale and that's asking for a problem.
The other problem is the input may also contain Cirillic letters that shouldn't be stripped or result in a PHP Notice.
echo iconv("utf-8", "ascii//TRANSLIT", "дом");
Is there a solution or do I have to create manually mapping of each character with diacritic to a one without it?
intl's Transliterator will let you define far more in-depth transliteration rules. The full documentation on transliteration rules can be found on icu-project.org.
$tests = [ "é", "дом" ];
$tl = Transliterator::create('Latin-ASCII;');
foreach($tests as $str) {
var_dump(
$tl->transliterate($str)
);
}
Output:
string(1) "e"
string(6) "дом"
The goal is to 'prevent collisions values already present in the table'? And there are accented letters that should be allowed to coexist with different accents and non-accents? Then change the collation of the PRIMARY
(or UNIQUE
) key that is causing the collisions.
Any ..._bin
COLLATION
will allow e
and é
to coexist (not collide during insertion) because it treats them as different.
Do you need ...general_ci
for some other reason? If so, please state the reason. If not, ALTER TABLE
to change the COLLATION
. I see no need for PHP code.