Fujitsu Logo
ENQUIRE

    Knowledge articles - Operation/Maintenance

    The message ERROR: input value length is x; too long for type <data-type> (y) is displayed when copying data from a CSV file to a table using the COPY FROM command.KB3001

    This may be because the CSV format file uses 0x00 as the NULL value.

    Resolution

    The COPY FROM command treats unquoted empty characters as NULL, rather than 0x00.

    If the CSV format file uses 0x00 as the NULL value, replace it with an empty string.

    If you want to distinguish NULL values from empty characters, you can also specify the string representing them in the NULL option of the COPY FROM command, so replace 0x00 with the specified string.

    For more information, refer to PostgreSQL documentation > Part VI - Reference > SQL commands > COPY

    Applicable to

    Product: FUJITSU Enterprise Postgres SE, FUJITSU Enterprise Postgres AE, PostgreSQL

    Architecture: X86, S390x

    Operating System: Windows, Linux

    Versions: from 9.5

    Can tables and indexes be stored in separate areas?KB3002

    Yes. By leveraging tablespaces, tables and indexes can be placed in separate areas.

    For details, refer to the sections below in PostgreSQL documentation:

    Applicable to

    Product: FUJITSU Enterprise Postgres SE, FUJITSU Enterprise Postgres AE, FUJITSU Enterprise Postgres for Kubernetes, PostgreSQL

    Architecture: X86, S390x

    Operating System: Windows, Linux

    Versions: from 9.5

    The database server's WAL space disk utilization is 100% and the database process has stopped. How do I solve it?KB3003

    Due to the I/O latency in the WAL archive storage area, the disk containing WAL is full and WAL cannot be written because the copy speed to the archive area is slower than the rate at which WAL data is generated.

    Resolution

    Consider the following:

    • Expand the capacity of the disk that store WAL files
    • Reduce the frequency of database update
    • Use faster disks for WAL archiving

    How to investigate

    • Check the error message in postgresql.log
      The log will contain an error message similar to the one below, stating that the server has stopped because the disk was full and that the system could not write WAL:
      PANIC: could not write to file "pg_wal/waltemp.4920": No space left on device
    • Compare the WAL generation rate and backup rate:
      • Check the WAL generation rate with timestamp intervals of each WAL file under the pg_wal directory.
      • Check the backup rate with timestamp intervals of each file under the archived_wal directory.
      • Confirm that the WAL generation rate is larger than backup rate

    As long as your environment can keep up with the average speed of the WAL generation of the server, the processing speed of the command for archiving is not important. Normal operations continue even if the archive process is slightly delayed, but note that a significantly slower archive process increases the amount of data lost in the event of a disaster. This also means that many segment files waiting to be archived will be stored in $PGDATA/pg_wal, which may cause the disk to be full. It is recommended that you monitor the archive process to ensure that it is working as intended.

    For more details, refer to PostgreSQL documentation > Part III - Server administration > Chapter 25 - Backup and restore > 25.3.1 - Setting up WAL archiving.

    You can also refer to our blog post How to solve the problem if pg_wal is full.

    Applicable to

    Product: FUJITSU Enterprise Postgres SE, FUJITSU Enterprise Postgres AE, FUJITSU Enterprise Postgres for Kubernetes, PostgreSQL

    Architecture: X86, S390x

    Operating System: Windows, Linux

    Versions: from 10

    What should I do if the database runs out of space to store backup data?KB3004

    Before clearing space in the backup data directory, one thing to note is the scenario where archiving log is enabled and archive logs are stored in the backup data directory. Once this destination runs out of space, the actual data directory will be filled with WAL logs. If that data directory runs out of space, the database may be unavailable. So, this must be urgently taken care of.

    If you run out of space in the backup data directory, the first step is to delete unnecessary files on it.

    If that does not solve the problem, do one of the following:

    • Temporarily save backup data to another location with sufficient space.
    • Replace the disk containing the backup data directory with a disk with more capacity.

    For more information, refer to FUJITSU Enterprise Postgres Operation Guide > Chapter 15 - Actions when an error occurs > 15.7 - Actions in response to insufficient space on the backup data storage destination [Linux version] [Windows version]

    Applicable to

    Product: FUJITSU Enterprise Postgres SE, FUJITSU Enterprise Postgres AE, PostgreSQL

    Architecture: X86, S390x

    Operating System: Windows, Linux

    Versions: from 9.5

    How to import data from a CSV file?KB3005

    You can import data into the database by using the COPY statement with the -c option of the psql command.

    Example:

    $ psql -d <db-name> -c "COPY table1 FROM 'import_data.csv' DELIMITER ','"

    For more information, refer to PostgreSQL documentation > Part VI - Reference > SQL Commands > COPY

    Applicable to

    Product: FUJITSU Enterprise Postgres SE, FUJITSU Enterprise Postgres AE, PostgreSQL

    Architecture: X86, S390x

    Operating System: Windows, Linux

    Versions: from 9.5

    Read our latest blogs

    Read our most recent articles regarding all aspects of PostgreSQL and FUJITSU Enterprise Postgres.