Multibyte strings php extension mbstring

How to enable mbstring in PHP?

It is required to have PHP enabled with mbstring (multi-byte string) support for storing and displaying multi-byte characters in PHPKB software. The installation wizard of the multi-language editions of PHPKB knowledge base software will automatically detect if you have PHP enabled with mbstring support. If it is not enabled, you may get an error that says:

Please refer to the tutorial below on how to enable mbstring on your server.

Why mbstring is required?

When we manipulate (trim, split, splice, etc.) strings encoded in a multi-byte encoding, we need to use special functions since two or more consecutive bytes may represent a single character in such encoding schemes. Otherwise, if we apply a non-multibyte-aware string function to the string, it probably fails to detect the beginning or end of the multibyte character and ends up with a corrupted garbage string that most likely loses its original meaning.

So, mbstring provides multibyte-specific string functions that help us deal with multibyte encodings in PHP. In addition to that, mbstring handles character encoding conversion between the possible encoding pairs. mbstring is designed to handle Unicode-based encodings such as UTF-8 and UCS-2 and many single-byte encodings for convenience.

MBstring Installation

Please note that «mbstring» is a non-default PHP extension. This means it is not enabled by default.

Installation on Linux Server

You can find out if this module is installed or not by looking at the list that is produced by this command

If you don’t have this installed on a Linux machine and if you have installed PHP from yum then run this command as a superuser. The command depends on your operating system.

sudo yum install php-mbstring
This command is true for all Red Hat Linux-based distributions using RPM and the ’yum’ package manager; this includes RHEL, Fedora, CentOS, Mandriva, and Intrinsic Linux, among others.
For Debian, Ubuntu distributions, please use the following command.
sudo apt-get install php-mbstring

After that, you will need to restart your Apache service using the following command:

Installation on Windows Server

If you have access to the php.ini, just remove the semicolon character (;) in front of the line below.

Once you have made the changes, don’t forget to restart your web server for the change to take effect.

Note: PHP extensions are not just enabled from php.ini file. First, you need to install that specific extension, i.e. place its DLL file «php-mbstring.dll» into the «/php/ext/» directory if it is not already present there. When you do that, make sure you have the correct build of DLL file. 32-bit, thread-safe PHP binary, built using VC9 for example, would only work with DLL files built using exact same tools and configuration: 32-bit, using VC9, with thread-safety turned on. And of course PHP API version also needs to match. If it doesn’t, once you restart the web server, you will receive ’Error 500 — Server error’ message.

Читайте также:  Java query list method

MBstring Configuration

Below is a sample excerpt from php.ini file which contains the configuration of mbstring variables.

[mbstring]
mbstring.language = all
mbstring.internal_encoding = UTF-8
mbstring.http_input = auto
mbstring.http_output = UTF-8
mbstring.encoding_translation = On
mbstring.detect_order = UTF-8
mbstring.substitute_character = none;
mbstring.func_overload = 0
mbstring.strict_encoding = Off

  • Applicable To: Standard (Multi-Language) Edition, Enterprise Multi-Language Edition (MySQL), Enterprise Multi-Language Edition (SQL Server)

35 people found this article helpful what about you?

Источник

Introduction

While there are many languages in which every necessary character can be represented by a one-to-one mapping to an 8-bit value, there are also several languages which require so many characters for written communication that they cannot be contained within the range a mere byte can code (A byte is made up of eight bits. Each bit can contain only two distinct values, one or zero. Because of this, a byte can only represent 256 unique values (two to the power of eight)). Multibyte character encoding schemes were developed to express more than 256 characters in the regular bytewise coding system.

When you manipulate (trim, split, splice, etc.) strings encoded in a multibyte encoding, you need to use special functions since two or more consecutive bytes may represent a single character in such encoding schemes. Otherwise, if you apply a non-multibyte-aware string function to the string, it probably fails to detect the beginning or ending of the multibyte character and ends up with a corrupted garbage string that most likely loses its original meaning.

mbstring provides multibyte specific string functions that help you deal with multibyte encodings in PHP. In addition to that, mbstring handles character encoding conversion between the possible encoding pairs. mbstring is designed to handle Unicode-based encodings such as UTF-8 and UCS-2 and many single-byte encodings for convenience (listed in Supported Character Encodings).

User Contributed Notes

  • Multibyte String
    • Introduction
    • Installing/Configuring
    • Predefined Constants
    • Summaries of supported encodings
    • Basics of Japanese multi-​byte encodings
    • HTTP Input and Output
    • Supported Character Encodings
    • Function Overloading Feature
    • PHP Character Encoding Requirements
    • Multibyte String Functions

    Источник

    References

    Please note that all the discussion about mb_str_replace in the comments is pretty pointless. str_replace works just fine with multibyte strings:

    $string = ‘漢字はユニコード’ ;
    $needle = ‘は’ ;
    $replace = ‘Foo’ ;

    echo str_replace ( $needle , $replace , $string );
    // outputs: 漢字Fooユニコード

    ?>

    The usual problem is that the string is evaluated as binary string, meaning PHP is not aware of encodings at all. Problems arise if you are getting a value «from outside» somewhere (database, POST request) and the encoding of the needle and the haystack is not the same. That typically means the source code is not saved in the same encoding as you are receiving «from outside». Therefore the binary representations don’t match and nothing happens.

    PHP can input and output Unicode, but a little different from what Microsoft means: when Microsoft says «Unicode», it unexplicitly means little-endian UTF-16 with BOM(FF FE = chr(255).chr(254)), whereas PHP’s «UTF-16» means big-endian with BOM. For this reason, PHP does not seem to be able to output Unicode CSV file for Microsoft Excel. Solving this problem is quite simple: just put BOM infront of UTF-16LE string.

    $unicode_str_for_Excel = chr(255).chr(254).mb_convert_encoding( $utf8_str, ‘UTF-16LE’, ‘UTF-8’);

    SOME multibyte encodings can safely be used in str_replace() and the like, others cannot. It’s not enough to ensure that all the strings involved use the same encoding: obviously they have to, but it’s not enough. It has to be the right sort of encoding.

    UTF-8 is one of the safe ones, because it was designed to be unambiguous about where each encoded character begins and ends in the string of bytes that makes up the encoded text. Some encodings are not safe: the last bytes of one character in a text followed by the first bytes of the next character may together make a valid character. str_replace() knows nothing about «characters», «character encodings» or «encoded text». It only knows about the string of bytes. To str_replace(), two adjacent characters with two-byte encodings just looks like a sequence of four bytes and it’s not going to know it shouldn’t try to match the middle two bytes.

    While real-world examples can be found of str_replace() mangling text, it can be illustrated by using the HTML-ENTITIES encoding. It’s not one of the safe ones. All of the strings being passed to str_replace() are valid HTML-ENTITIES-encoded text so the «all inputs use the same encoding» rule is satisfied.

    $string = ‘x<y’ ;
    mb_internal_encoding ( ‘HTML-ENTITIES’ );

    echo «Text length: » , mb_strlen ( $string ), «\tString length: » , strlen ( $string ), » . » , $string , «\n» ;
    // Three characters, six bytes; the text reads «x

    $newstring = str_replace ( ‘l’ , ‘g’ , $string );
    echo «Text length: » , mb_strlen ( $newstring ), «\tString length: » , strlen ( $newstring ), » . » , $newstring , «\n» ;
    // Three characters, six bytes, but now the text reads «x>y»; the wrong characters have changed.

    $newstring = str_replace ( ‘;’ , ‘:’ , $string );
    echo «Text length: » , mb_strlen ( $newstring ), «\tString length: » , strlen ( $newstring ), » . » , $newstring , «\n» ;
    // Now even the length of the text is wrong and the text is trashed.

    ?>

    Even though neither ‘l’ nor ‘;’ appear in the text «xy» and in the other it broke the encoding completely.

    One more reason to use UTF-8 if you can, I guess.

    A small note for those who will follow rawsrc at gmail dot com’s advice: mb_split uses regular expressions, in which case it may make sense to use built-in function mb_ereg_replace.

    Note that some of the multi-byte functions run in O(n) time, rather than constant time as is the case for their single-byte equivalents. This includes any functionality requiring access at a specific index, since random access is not possible in a string whose number of bytes will not necessarily match the number of characters. Affected functions include: mb_substr(), mb_strstr(), mb_strcut(), mb_strpos(), etc.

    function mb_str_pad ( $input , $pad_length , $pad_string , $pad_style , $encoding = «UTF-8» ) <
    return str_pad ( $input ,
    strlen ( $input )- mb_strlen ( $input , $encoding )+ $pad_length , $pad_string , $pad_style );
    >
    ?>

    Yet another single-line mb_trim() function

    function mb_trim ( $string , $trim_chars = ‘\s’ ) return preg_replace ( ‘/^[‘ . $trim_chars . ‘]*(?U)(.*)[‘ . $trim_chars . ‘]*$/u’ , ‘\\1’ , $string );
    >
    $string = ‘ «some text.» ‘ ;
    echo mb_trim ( $string , ‘\s».’ );
    //some text
    ?>

    This would be one way to create a multibyte substr_replace function

    function mb_substr_replace ( $output , $replace , $posOpen , $posClose ) <
    return mb_substr ( $output , 0 , $posOpen ). $replace . mb_substr ( $output , $posClose + 1 );
    >
    ?>

    str_replace is NOT multi-bite safe.

    This Ukrainian word gives a bug when used in the next code: відео

    $result = str_replace(str_split($rubishcharacters), ‘ ‘, $searchstring);

    PHP5 has no mb_trim(), so here’s one I made. It work just as trim(), but with the added bonus of PCRE character classes (including, of course, all the useful Unicode ones such as \pZ).

    Unlike other approaches that I’ve seen to this problem, I wanted to emulate the full functionality of trim() — in particular, the ability to customise the character list.

    /**
    * Trim characters from either (or both) ends of a string in a way that is
    * multibyte-friendly.
    *
    * Mostly, this behaves exactly like trim() would: for example supplying ‘abc’ as
    * the charlist will trim all ‘a’, ‘b’ and ‘c’ chars from the string, with, of
    * course, the added bonus that you can put unicode characters in the charlist.
    *
    * We are using a PCRE character-class to do the trimming in a unicode-aware
    * way, so we must escape ^, \, — and ] which have special meanings here.
    * As you would expect, a single \ in the charlist is interpretted as
    * «trim backslashes» (and duly escaped into a double-\ ). Under most circumstances
    * you can ignore this detail.
    *
    * As a bonus, however, we also allow PCRE special character-classes (such as ‘\s’)
    * because they can be extremely useful when dealing with UCS. ‘\pZ’, for example,
    * matches every ‘separator’ character defined in Unicode, including non-breaking
    * and zero-width spaces.
    *
    * It doesn’t make sense to have two or more of the same character in a character
    * class, therefore we interpret a double \ in the character list to mean a
    * single \ in the regex, allowing you to safely mix normal characters with PCRE
    * special classes.
    *
    * *Be careful* when using this bonus feature, as PHP also interprets backslashes
    * as escape characters before they are even seen by the regex. Therefore, to
    * specify ‘\\s’ in the regex (which will be converted to the special character
    * class ‘\s’ for trimming), you will usually have to put *4* backslashes in the
    * PHP code — as you can see from the default value of $charlist.
    *
    * @param string
    * @param charlist list of characters to remove from the ends of this string.
    * @param boolean trim the left?
    * @param boolean trim the right?
    * @return String
    */
    function mb_trim ( $string , $charlist = ‘\\\\s’ , $ltrim = true , $rtrim = true )
    <
    $both_ends = $ltrim && $rtrim ;

    if( $both_ends )
    <
    $pattern_middle = $left_pattern . ‘|’ . $right_pattern ;
    >
    elseif( $ltrim )
    <
    $pattern_middle = $left_pattern ;
    >
    else
    <
    $pattern_middle = $right_pattern ;
    >

    return preg_replace ( «/ $pattern_middle /usSD» , » , $string ) );
    >
    ?>

    Источник

Оцените статью