arsvendg/Stirling-PDF
1
0
Fork 0
mirror of https://github.com/arsvendg/Stirling-PDF.git synced 2026-07-16 19:33:11 +02:00
Code Issues Actions 4 Packages Projects Releases 1 Wiki Activity

fix(translations): improve translation merger CLI and sync missing UI strings across locales (#5309)

Browse Source 
# Description of Changes

This pull request updates the Arabic translation file
(`frontend/public/locales/ar-AR/translation.toml`) with a large number
of new and improved strings, adding support for new features and
enhancing clarity and coverage across the application. Additionally, it
makes several improvements to the TOML language check script
(`.github/scripts/check_language_toml.py`) and updates the corresponding
GitHub Actions workflow to better track and validate translation
changes.

**Translation updates and enhancements:**

* Added translations for new features and UI elements, including
annotation tools, PDF/A-3b conversion, line art compression, background
removal, split modes, onboarding tours, and more.
[[1]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR343-R346)
[[2]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR442-R460)
[[3]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR514-R523)
[[4]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR739-R743)
[[5]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR1281-R1295)
[[6]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR1412-R1416)
[[7]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR2362-R2365)
[[8]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR2411-R2415)
[[9]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR2990)
[[10]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR3408-R3420)
[[11]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR3782-R3794)
[[12]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR3812-R3815)
[[13]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR3828-R3832)
[[14]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effL3974-R4157)
[[15]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR4208-R4221)
[[16]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR5247)
[[17]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR5414-R5423)
[[18]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR5444-R5447)
* Improved and expanded coverage for settings, security, onboarding, and
help menus, including detailed descriptions and tooltips for new and
existing features.
[[1]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR442-R460)
[[2]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR5247)
[[3]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR5414-R5423)
[[4]](diffhunk://#diff-460d5f61a7649a5b149373af2e52a8a87d9a1964cf54240a78ad4747e7233effR5444-R5447)

**TOML language check script improvements:**

* Increased the maximum allowed TOML file size from 500 KB to 570 KB to
accommodate larger translation files.
* Improved file validation logic to more accurately skip or process
files based on directory structure and file type, and added informative
print statements for skipped files.
* Enhanced reporting in the difference check: now, instead of raising
exceptions for unsafe files or oversized files, the script logs warnings
and continues processing, improving robustness and clarity in CI
reports.
* Adjusted the placement of file check report lines for clarity in the
generated report.

**Workflow and CI improvements:**

* Updated the GitHub Actions workflow
(`.github/workflows/check_toml.yml`) to trigger on changes to the
translation script and workflow files, in addition to translation TOMLs,
ensuring all relevant changes are validated.

These changes collectively improve the translation quality and coverage
for Arabic users, enhance the reliability and clarity of the translation
validation process, and ensure smoother CI/CD workflows for localization
updates.

<img width="654" height="133" alt="image"
src="https://github.com/user-attachments/assets/9f3e505d-927f-4dc0-9098-cee70bbe85ca"
/>


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
This commit is contained in:
Ludy
2026-01-14 00:31:05 +00:00
committed by GitHub
parent db049a3467
commit 472ee54098
46 changed files with 16654 additions and 2060 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/counter_translation_v3.py
+118 -122
View File
@@ -1,16 +1,16 @@
"""
A script to update language progress status in README.md based on
properties file comparison.
frontend locale TOML file comparisons.
This script compares the default (reference) properties file, usually
`messages_en_GB.properties`, with other translation files in the
`app/core/src/main/resources/` directory.
It determines how many lines are fully translated and automatically updates
This script compares the default (reference) TOML file,
`frontend/public/locales/en-GB/translation.toml`, with other translation
files in `frontend/public/locales/*/translation.toml`.
It determines how many keys are fully translated and automatically updates
progress badges in the `README.md`.
Additionally, it maintains a TOML configuration file
(`scripts/ignore_translation.toml`) that defines which keys are ignored
during comparison (e.g., static values like `language.direction`).
during comparison (e.g., values intentionally matching English).
Author: Ludy87
@@ -18,31 +18,31 @@ Usage:
Run this script directly from the project root.
# --- Compare all translation files and update README.md ---
$ python scripts/counter_translation.py
$ python scripts/counter_translation_v3.py
This will:
• Compare all files matching messages_*.properties
• Compare all files matching frontend/public/locales/*/translation.toml
• Update progress badges in README.md
• Update/format ignore_translation.toml automatically
# --- Check a single language file ---
$ python scripts/counter_translation.py --lang messages_fr_FR.properties
$ python scripts/counter_translation_v3.py --lang fr-FR
This will:
• Compare the French translation file against the English reference
• Print the translation percentage in the console
# --- Print ONLY the percentage (for CI pipelines or automation) ---
$ python scripts/counter_translation.py --lang messages_fr_FR.properties --show-percentage
$ python scripts/counter_translation_v3.py --lang fr-FR --show-percentage
Example output:
87
Arguments:
-l, --lang <file> Specific properties file to check
(relative or absolute path).
--show-percentage Print only the percentage (no formatting, ideal for CI/CD).
--show-missing-keys Show the list of missing keys when checking a single language file.
-l, --lang <locale or file> Specific locale to check (e.g. 'de-DE'),
a directory, or a full path to translation.toml.
--show-percentage Print only the percentage (no formatting, ideal for CI/CD).
--show-missing-keys Show the list of missing keys when checking a single language file.
"""
import argparse
@@ -50,10 +50,18 @@ import glob
import os
import re
import sys
from collections.abc import Mapping
from typing import Iterable
import tomlkit
import tomlkit.toml_file
# Ensure tomlkit is installed before importing
try:
import tomlkit
except ImportError:
raise ImportError(
"The 'tomlkit' library is not installed. Please install it using 'pip install tomlkit'."
)
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
def convert_to_multiline(data: tomlkit.TOMLDocument) -> tomlkit.TOMLDocument:
@@ -102,7 +110,10 @@ def write_readme(progress_list: list[tuple[str, int]]) -> None:
Returns:
None
"""
with open("README.md", encoding="utf-8") as file:
with open(
os.path.join(os.getcwd(), "devGuide", "HowToAddNewLanguage.md"),
encoding="utf-8",
) as file:
content = file.readlines()
for i, line in enumerate(content[2:], start=2):
@@ -115,56 +126,62 @@ def write_readme(progress_list: list[tuple[str, int]]) -> None:
f"![{value}%](https://geps.dev/progress/{value})",
)
with open("README.md", "w", encoding="utf-8", newline="\n") as file:
with open(
os.path.join(os.getcwd(), "devGuide", "HowToAddNewLanguage.md"),
"w",
encoding="utf-8",
newline="\n",
) as file:
file.writelines(content)
def load_reference_keys(default_file_path: str) -> set[str]:
"""Reads all keys from the reference properties file (excluding comments and empty lines).
This function skips the first 5 lines (assumed to be headers or metadata) and then
extracts keys from lines containing '=' separators, ignoring comments (#) and empty lines.
It also handles potential BOM (Byte Order Mark) characters.
def _flatten_toml(data: Mapping[str, object], prefix: str = "") -> dict[str, object]:
"""Flattens a TOML document into dotted keys for comparison.
Args:
default_file_path (str): The path to the default (reference) properties file.
data (Mapping[str, object]): TOML content loaded into a mapping.
prefix (str): Prefix for nested keys.
Returns:
set[str]: A set of unique keys found in the reference file.
dict[str, object]: Flattened key/value mapping.
"""
keys: set[str] = set()
with open(default_file_path, encoding="utf-8") as f:
# Skip the first 5 lines (headers)
for _ in range(5):
try:
next(f)
except StopIteration:
break
flattened: dict[str, object] = {}
for key, value in data.items():
combined_key = f"{prefix}{key}"
if isinstance(value, Mapping):
flattened.update(_flatten_toml(value, f"{combined_key}."))
else:
flattened[combined_key] = value
return flattened
for line in f:
s = line.strip()
if not s or s.startswith("#") or "=" not in s:
continue
k, _ = s.split("=", 1)
keys.add(k.strip().replace("\ufeff", "")) # BOM protection
return keys
def load_translation_entries(file_path: str) -> dict[str, object]:
"""Reads and flattens translation entries from a TOML file.
Args:
file_path (str): Path to translation.toml.
Returns:
dict[str, object]: Flattened key/value entries.
"""
with open(file_path, encoding="utf-8") as f:
document = tomlkit.parse(f.read())
return _flatten_toml(document)
def _lang_from_path(file_path: str) -> str:
"""Extracts the language code from a properties file path.
"""Extracts the language code from a locale TOML file path.
Assumes the filename format is 'messages_<language>.properties', where <language>
is the code like 'fr_FR'.
Assumes the filename format is '<locale>/translation.toml', where <locale>
is the code like 'fr-FR'.
Args:
file_path (str): The full path to the properties file.
file_path (str): The full path to the TOML translation file.
Returns:
str: The extracted language code.
"""
return (
os.path.basename(file_path).split("messages_", 1)[1].split(".properties", 1)[0]
)
return os.path.basename(os.path.dirname(file_path))
def compare_files(
@@ -174,16 +191,16 @@ def compare_files(
show_missing_keys: bool = False,
show_percentage: bool = False,
) -> list[tuple[str, int]]:
"""Compares the default properties file with other properties files in the directory.
"""Compares the default TOML file with other locale TOML files in the directory.
This function calculates translation progress for each language file by comparing
keys and values line-by-line, skipping headers. It accounts for ignored keys defined
in a TOML configuration file and updates that file with cleaned ignore lists.
English variants (en_GB, en_US) are hardcoded to 100% progress.
keys and values. It accounts for ignored keys defined in a TOML configuration file
and updates that file with cleaned ignore lists. English variants (en-GB, en-US)
are hardcoded to 100% progress.
Args:
default_file_path (str): The path to the default properties file (reference).
file_paths (Iterable[str]): Iterable of paths to properties files to compare.
default_file_path (str): The path to the default TOML file (reference).
file_paths (Iterable[str]): Iterable of paths to TOML files to compare.
ignore_translation_file (str): Path to the TOML file with ignore/missing configurations per language.
show_missing_keys (bool, optional): If True, prints the list of missing keys for each file. Defaults to False.
show_percentage (bool, optional): If True, suppresses detailed output and focuses on percentage calculation. Defaults to False.
@@ -192,14 +209,9 @@ def compare_files(
list[tuple[str, int]]: A sorted list of tuples containing language codes and progress percentages
(descending order by percentage). Duplicates are removed.
"""
# Count total translatable lines in reference (excluding empty and comments)
num_lines = sum(
1
for line in open(default_file_path, encoding="utf-8")
if line.strip() and not line.strip().startswith("#")
)
ref_keys: set[str] = load_reference_keys(default_file_path)
reference_entries = load_translation_entries(default_file_path)
ref_keys = set(reference_entries.keys())
num_lines = len(ref_keys)
result_list: list[tuple[str, int]] = []
sort_ignore_translation: tomlkit.TOMLDocument
@@ -215,10 +227,12 @@ def compare_files(
language = _lang_from_path(file_path)
# Hardcode English variants to 100%
if "en_GB" in language or "en_US" in language:
if language in {"en-GB", "en-US"}:
result_list.append((language, 100))
continue
language = language.replace("-", "_")
# Initialize language table in TOML if missing
if language not in sort_ignore_translation:
sort_ignore_translation[language] = tomlkit.table()
@@ -239,58 +253,30 @@ def compare_files(
if key in ref_keys or key == "language.direction"
]
translation_entries = load_translation_entries(file_path)
fails = 0
missing_str_keys: list[str] = []
with (
open(default_file_path, encoding="utf-8") as default_file,
open(file_path, encoding="utf-8") as file,
):
# Skip headers (first 5 lines) in both files
for _ in range(5):
next(default_file)
try:
next(file)
except StopIteration:
fails = num_lines
break
for line_num, (line_default, line_file) in enumerate(
zip(default_file, file), start=6
for default_key, default_value in reference_entries.items():
if default_key not in translation_entries:
fails += 1
missing_str_keys.append(default_key)
continue
file_value = translation_entries[default_key]
if (
default_value == file_value
and default_key not in sort_ignore_translation[language]["ignore"]
):
try:
# Ignoring empty lines and lines starting with #
if line_default.strip() == "" or line_default.startswith("#"):
# Missing translation (same as default and not ignored)
fails += 1
missing_str_keys.append(default_key)
if default_value != file_value:
if default_key in sort_ignore_translation[language]["ignore"]:
if default_key == "language.direction":
continue
default_key, default_value = line_default.split("=", 1)
file_key, file_value = line_file.split("=", 1)
default_key = default_key.strip()
default_value = default_value.strip()
file_key = file_key.strip()
file_value = file_value.strip()
if (
default_value == file_value
and default_key
not in sort_ignore_translation[language]["ignore"]
):
# Missing translation (same as default and not ignored)
fails += 1
missing_str_keys.append(default_key)
if default_value != file_value:
if default_key in sort_ignore_translation[language]["ignore"]:
# Remove from ignore if actually translated
sort_ignore_translation[language]["ignore"].remove(
default_key
)
except ValueError as e:
print(f"Error processing line {line_num} in {file_path}: {e}")
print(f"{line_default}|{line_file}")
sys.exit(1)
except IndexError:
# Handle mismatched line counts
fails += 1
continue
# Remove from ignore if actually translated
sort_ignore_translation[language]["ignore"].remove(default_key)
if show_missing_keys:
if len(missing_str_keys) > 0:
@@ -327,19 +313,19 @@ def main() -> None:
(with optional percentage output) or all files and updates the README.md.
Command-line options:
--lang, -l <file>: Specific properties file to check (e.g., 'messages_fr_FR.properties').
--lang, -l <file>: Specific locale to check, e.g. 'fr-FR'
--show-percentage: Print only the translation percentage for --lang and exit.
--show-missing-keys: Show the list of missing keys when checking a single language file.
"""
parser = argparse.ArgumentParser(
description="Compare i18n property files and optionally update README badges."
description="Compare frontend i18n TOML files and optionally update README badges."
)
parser.add_argument(
"--lang",
"-l",
help=(
"Specific properties file to check, e.g. 'messages_fr_FR.properties'. "
"If a relative filename is given, it is resolved against the resources directory."
"Specific locale to check, e.g. 'fr-FR'. "
"If a relative filename is given, it is resolved against the locales directory."
),
)
parser.add_argument(
@@ -359,8 +345,8 @@ def main() -> None:
# Project layout assumptions
cwd = os.getcwd()
resources_dir = os.path.join(cwd, "app", "core", "src", "main", "resources")
reference_file = os.path.join(resources_dir, "messages_en_GB.properties")
locales_dir = os.path.join(cwd, "frontend", "public", "locales")
reference_file = os.path.join(locales_dir, "en-GB", "translation.toml")
scripts_directory = os.path.join(cwd, "scripts")
translation_state_file = os.path.join(scripts_directory, "ignore_translation.toml")
@@ -370,7 +356,19 @@ def main() -> None:
if os.path.isabs(lang_input) or os.path.exists(lang_input):
lang_file = lang_input
else:
lang_file = os.path.join(resources_dir, lang_input)
candidate = os.path.join(locales_dir, lang_input)
candidate_with_file = os.path.join(
locales_dir, lang_input, "translation.toml"
)
if os.path.exists(candidate):
if os.path.isdir(candidate):
lang_file = candidate_with_file
else:
lang_file = candidate
elif os.path.exists(candidate_with_file):
lang_file = candidate_with_file
else:
lang_file = lang_input
if not os.path.exists(lang_file):
print(f"ERROR: Could not find language file: {lang_file}")
@@ -384,7 +382,7 @@ def main() -> None:
args.show_percentage,
)
# Find the exact tuple for the requested language
wanted_key = _lang_from_path(lang_file)
wanted_key = _lang_from_path(lang_file).replace("-", "_")
for lang, pct in results:
if lang == wanted_key:
if args.show_percentage:
@@ -400,13 +398,11 @@ def main() -> None:
sys.exit(3)
# Default behavior (no --lang): process all and update README
messages_file_paths = glob.glob(
os.path.join(resources_dir, "messages_*.properties")
)
messages_file_paths = glob.glob(os.path.join(locales_dir, "*", "translation.toml"))
progress = compare_files(
reference_file, messages_file_paths, translation_state_file
)
write_readme(progress)
# write_readme(progress)
if __name__ == "__main__":