Enhance cleanup dialog with sensitivity slider for problematic entries

We don't want to display utc

README.md

@@ -1,16 +1,89 @@
-# journaler
+# Journaler
 
-A new Flutter project.
+![Journaler Logo](assets/app_icon.ico)
 
-## Getting Started
+Journaler is a "cross-platform" desktop application designed to encourage regular journaling by providing automated reminders and a clean writing environment.
 
-This project is a starting point for a Flutter application.
+## Overview
 
-A few resources to get you started if this is your first Flutter project:
+Journaler helps you build a consistent journaling habit by periodically reminding you to record your thoughts throughout the day. It runs silently in the system tray and pops up at customizable intervals, making it perfect for:
 
-- [Lab: Write your first Flutter app](https://docs.flutter.dev/get-started/codelab)
+- **Daily reflection and mindfulness practice**
-- [Cookbook: Useful Flutter samples](https://docs.flutter.dev/cookbook)
+- **Tracking your thoughts and activities**
+- **Building a journaling habit**
+- **Capturing ideas before they fade**
 
-For help getting started with Flutter development, view the
+![Journaler Screenshot](docs/screenshots/journaler_main.png)
-[online documentation](https://docs.flutter.dev/), which offers tutorials,
+
-samples, guidance on mobile development, and a full API reference.
+## Features
+
+- **Automated Reminders**: Customizable popup intervals to remind you to journal
+- **System Tray Integration**: Runs silently in the background
+- **Scratch Pad**: Quick notes area for temporary thoughts
+- **Journal History**: Browse through past entries
+- **Full-Text Search**: Find entries containing specific words or phrases
+- **Dark/Light Theme**: Automatically follows your system theme
+- **Notification Sounds**: Audio cues when it's time to journal
+- **Single Instance**: Only one copy of the app runs at a time
+
+## Usage
+
+### Getting Started
+
+1. **First Launch**: When you first launch Journaler, it will start in the system tray
+2. **Writing Entries**: The app will automatically pop up at set intervals (default: 20 minutes)
+3. **Manual Access**: Click the system tray icon to show the app manually
+4. **"Multiple" instances**: Running another instance will make the existing instance show itself
+5. **Saving entries**: Entries are saved automatically when the popup is closed either manually or by hitting the escape key
+
+### Main Interface
+
+1. **Current Entry**: Write your thoughts for the current moment
+2. **Previous Entry**: View what you wrote last time
+3. **Scratch Pad**: Space for quick notes that persist between sessions
+4. **Navigation**: Browse through your past entries
+5. **Search**: Find specific content in your journal history (May also be invoked by CTRL-F)
+
+![Journaler Interface](docs/screenshots/journaler_annotated.png)
+
+### Settings
+
+You can adjust:
+
+- **Popup Interval**: Change how frequently Journaler reminds you
+- **Notification Sound**: Select your preferred audio reminder
+- **Volume**: Adjust the sound level of reminders
+
+## Data privacy
+
+All your journal entries are stored securely on your local machine:
+
+- **Windows**: `C:\Users\YourUsername\.journaler\`
+
+No network requests are made, no data sent or received, your data is yours alone.
+
+Data is stored in an SQLite database with a very simple schema that can be fucked with easily.
+
+## Development
+
+This application is built with Flutter. If you want to build from source:
+
+1. Install Flutter (version 3.7.2 or higher)
+2. Clone the repository
+3. Run `flutter pub get` to install dependencies
+4. Use `flutter run` to run the application in debug mode
+
+## Known issues
+
+I don't know whether or not the sound customization works.<br>
+In theory only 2 sound files are bundled with the app so, in theory, you may only choose between those two.<br>
+But I don't know how flutter handles assets, currently I am not very motivated to figure this out.
+
+Scrolling through long notes is difficult since we're intercepting the scroll event and using it to scroll through notes themselves.<br>
+This, again, is not really an issue for me so I'm not very motivated to fix it.
+
+Showing and focusing the window is really annoying for "gamers" and generally whoever is doing anything at the time.<br>
+But I have no better ideas.<br>
+If we show the window without focusing it, it will still be on top and, if not focused, will be even more annoying because to close it you have to click on it (to focus it) and THEN close it.<br>
+If we only play the sound as a reminder we could miss it or simply get used to the sound and learn to ignore it.<br>
+I have no ideas for a better solution.

lib/db.dart

@@ -6,10 +6,23 @@ import 'package:sqflite_common_ffi/sqflite_ffi.dart';
 const settingsDir = '.journaler';
 const dbFileName = 'journaler.db';
 
+// Add this at the top level
+typedef ShowMessageCallback = void Function(String message);
+
 class DB {
   static late Database db;
+  static bool _hasIcuSupport = false;
+  static ShowMessageCallback? _showMessage;
 
-  static const String _schema = '''
+  // Add this to register the callback
+  static void registerMessageCallback(ShowMessageCallback callback) {
+    _showMessage = callback;
+  }
+
+  // Add this method to check ICU status
+  static bool get hasIcuSupport => _hasIcuSupport;
+
+  static const String _baseSchema = '''
 CREATE TABLE IF NOT EXISTS notes (
   id INTEGER PRIMARY KEY AUTOINCREMENT,
   date TEXT DEFAULT CURRENT_TIMESTAMP,
@@ -17,10 +30,7 @@ CREATE TABLE IF NOT EXISTS notes (
 );
 CREATE INDEX IF NOT EXISTS idx_notes_date ON notes (date);
 CREATE UNIQUE INDEX IF NOT EXISTS idx_notes_date_unique ON notes (date);
--- Todos are "static", we are only interested in the latest entry
+
--- ie. they're not really a list but instead a string
--- But we will also keep a history of all todos
--- Because we might as well
 CREATE TABLE IF NOT EXISTS scratches (
   id INTEGER PRIMARY KEY AUTOINCREMENT,
   date TEXT DEFAULT CURRENT_TIMESTAMP,
@@ -35,6 +45,163 @@ CREATE TABLE IF NOT EXISTS settings (
 );
 ''';
 
+  static const String _ftsSchemaWithIcu = '''
+-- Create virtual FTS5 table with Unicode tokenizer
+CREATE VIRTUAL TABLE IF NOT EXISTS notes_fts USING fts5(
+  content,
+  date,
+  content='notes',
+  content_rowid='id',
+  tokenize='unicode61 remove_diacritics 2 tokenchars "\u0401\u0451\u0410-\u044f"'
+);
+''';
+
+  static const String _ftsSchemaBasic = '''
+-- Create virtual FTS5 table with basic tokenizer
+CREATE VIRTUAL TABLE IF NOT EXISTS notes_fts USING fts5(
+  content,
+  date,
+  content='notes',
+  content_rowid='id',
+  tokenize='unicode61'
+);
+''';
+
+  static const String _ftsTriggers = '''
+-- Trigger to keep FTS table in sync with notes table when inserting
+CREATE TRIGGER IF NOT EXISTS notes_ai AFTER INSERT ON notes BEGIN
+  INSERT INTO notes_fts(rowid, content, date) VALUES (new.id, new.content, new.date);
+END;
+
+-- Trigger to keep FTS table in sync when deleting notes
+CREATE TRIGGER IF NOT EXISTS notes_ad AFTER DELETE ON notes BEGIN
+  DELETE FROM notes_fts WHERE rowid = old.id;
+END;
+
+-- Trigger to keep FTS table in sync when updating notes
+CREATE TRIGGER IF NOT EXISTS notes_au AFTER UPDATE ON notes BEGIN
+  UPDATE notes_fts SET content = new.content, date = new.date WHERE rowid = old.id;
+END;
+''';
+
+  // Add this method to check ICU status with visible feedback
+  static Future<bool> checkAndShowIcuStatus() async {
+    final status = _hasIcuSupport;
+    final message =
+        status
+            ? 'ICU support is enabled - Full Unicode search available'
+            : 'ICU support is not available - Unicode search will be limited\n'
+                'Looking for sqlite3_icu.dll in the application directory';
+    debugPrint(message);
+    _showMessage?.call(message);
+    return status;
+  }
+
+  static Future<bool> _checkIcuSupport(Database db) async {
+    try {
+      debugPrint(
+        '\n================== CHECKING UNICODE SUPPORT ==================',
+      );
+
+      // Test Unicode support with a simple query
+      try {
+        debugPrint('Testing Unicode support...');
+        // Test with some Cyrillic characters
+        await db.rawQuery("SELECT 'тест' LIKE '%ест%'");
+        await db.rawQuery("SELECT 'ТЕСТ' LIKE '%ЕСТ%'");
+
+        final message = 'Unicode support is available';
+        debugPrint(message);
+        _showMessage?.call(message);
+        return true;
+      } catch (e, stackTrace) {
+        final message = 'Unicode support test failed';
+        debugPrint('$message:');
+        debugPrint('Error: $e');
+        debugPrint('Stack trace: $stackTrace');
+
+        // Try to get SQLite version and compile options for debugging
+        try {
+          final version = await db.rawQuery('SELECT sqlite_version()');
+          final compileOpts = await db.rawQuery('PRAGMA compile_options');
+          debugPrint('SQLite version: $version');
+          debugPrint('SQLite compile options: $compileOpts');
+        } catch (e) {
+          debugPrint('Could not get SQLite info: $e');
+        }
+
+        _showMessage?.call(message);
+        return false;
+      }
+    } catch (e, stackTrace) {
+      final message = 'Failed to test Unicode support';
+      debugPrint('$message:');
+      debugPrint('Error: $e');
+      debugPrint('Stack trace: $stackTrace');
+      _showMessage?.call(message);
+      return false;
+    } finally {
+      debugPrint('=====================================================\n');
+    }
+  }
+
+  static Future<void> _recreateFtsTable(Database db, bool useIcu) async {
+    debugPrint('Updating FTS table configuration...');
+
+    try {
+      // Start a transaction to ensure data safety
+      await db.transaction((txn) async {
+        // First, create a temporary table with the new configuration
+        final tempTableName = 'notes_fts_temp';
+        final schema = useIcu ? _ftsSchemaWithIcu : _ftsSchemaBasic;
+
+        // Create temp table with new configuration
+        await txn.execute(schema.replaceAll('notes_fts', tempTableName));
+
+        // Copy data from old FTS table if it exists
+        try {
+          debugPrint('Copying existing FTS data to temporary table...');
+          await txn.execute('''
+            INSERT INTO $tempTableName(rowid, content, date)
+            SELECT rowid, content, date FROM notes_fts
+          ''');
+          debugPrint('Data copied successfully');
+        } catch (e) {
+          debugPrint('No existing FTS data to copy: $e');
+        }
+
+        // Drop old triggers
+        debugPrint('Updating triggers...');
+        await txn.execute('DROP TRIGGER IF EXISTS notes_ai');
+        await txn.execute('DROP TRIGGER IF EXISTS notes_ad');
+        await txn.execute('DROP TRIGGER IF EXISTS notes_au');
+
+        // Drop old FTS table
+        await txn.execute('DROP TABLE IF EXISTS notes_fts');
+
+        // Rename temp table to final name
+        await txn.execute('ALTER TABLE $tempTableName RENAME TO notes_fts');
+
+        // Create new triggers
+        await txn.execute(_ftsTriggers);
+
+        // Rebuild FTS index from notes table to ensure consistency
+        debugPrint('Rebuilding FTS index from notes table...');
+        await txn.execute('''
+          INSERT OR REPLACE INTO notes_fts(rowid, content, date)
+          SELECT id, content, date FROM notes
+        ''');
+
+        debugPrint('FTS table update completed successfully');
+      });
+    } catch (e, stackTrace) {
+      debugPrint('Error updating FTS table:');
+      debugPrint('Error: $e');
+      debugPrint('Stack trace: $stackTrace');
+      rethrow;
+    }
+  }
+
   static Future<String> _getDatabasePath() async {
     debugPrint('Attempting to get database path...');
     if (Platform.isWindows || Platform.isLinux) {
@@ -44,7 +211,7 @@ CREATE TABLE IF NOT EXISTS settings (
       if (home == null) {
         throw Exception('Could not find home directory');
       }
-      debugPrint('Home directory found: home');
+      debugPrint('Home directory found: $home');
 
       final dbDir = Directory(path.join(home, settingsDir));
       if (!await dbDir.exists()) {
@@ -58,33 +225,107 @@ CREATE TABLE IF NOT EXISTS settings (
     } else {
       // Default path for other platforms
       final databasesPath = await databaseFactoryFfi.getDatabasesPath();
-      debugPrint('Using default databases path: databasesPath');
+      debugPrint('Using default databases path: $databasesPath');
       return path.join(databasesPath, dbFileName);
     }
   }
 
   static Future<void> init() async {
     debugPrint('Starting database initialization...');
+
+    // Initialize SQLite FFI
     sqfliteFfiInit();
+    final databaseFactory = databaseFactoryFfi;
 
-    final dbPath = await _getDatabasePath();
+    // Create a temporary database to check version
-    debugPrint('Database path: dbPath');
-
     try {
-      db = await databaseFactoryFfi.openDatabase(
+      debugPrint(
-        dbPath,
+        '\n================== SQLITE VERSION CHECK ==================',
+      );
+      final tempDb = await databaseFactory.openDatabase(
+        ':memory:',
         options: OpenDatabaseOptions(
           version: 1,
           onCreate: (db, version) async {
-            debugPrint('Creating database schema...');
+            final results = await db.rawQuery('SELECT sqlite_version()');
-            await db.execute(_schema);
+            debugPrint('SQLite version: ${results.first.values.first}');
-            debugPrint('Database schema created successfully');
+
+            final compileOpts = await db.rawQuery('PRAGMA compile_options');
+            debugPrint('SQLite compile options:');
+            for (var opt in compileOpts) {
+              debugPrint('  ${opt.values.first}');
+            }
           },
         ),
       );
-      debugPrint('Database opened and initialized');
+      await tempDb.close();
-    } catch (e) {
+      debugPrint('=====================================================\n');
-      debugPrint('Failed to initialize database: e');
+    } catch (e, stackTrace) {
+      debugPrint('Error checking SQLite version:');
+      debugPrint('Error: $e');
+      debugPrint('Stack trace: $stackTrace');
+    }
+
+    await databaseFactory.setDatabasesPath(
+      await databaseFactory.getDatabasesPath(),
+    );
+
+    final dbPath = await _getDatabasePath();
+    debugPrint('Database path: $dbPath');
+
+    try {
+      db = await databaseFactory.openDatabase(
+        dbPath,
+        options: OpenDatabaseOptions(
+          version: 2,
+          onConfigure: (db) async {
+            debugPrint('Configuring database...');
+            await db.execute('PRAGMA foreign_keys = ON');
+            debugPrint('Database configured');
+          },
+          onCreate: (db, version) async {
+            debugPrint('Creating database schema...');
+            await db.execute(_baseSchema);
+
+            // Check for Unicode support on first creation
+            _hasIcuSupport = await _checkIcuSupport(db);
+            await _recreateFtsTable(db, _hasIcuSupport);
+
+            debugPrint('Database schema created successfully');
+          },
+          onOpen: (db) async {
+            debugPrint('Database opened, checking Unicode support...');
+            try {
+              _hasIcuSupport = await _checkIcuSupport(db);
+              debugPrint('Unicode support check completed: $_hasIcuSupport');
+            } catch (e, stackTrace) {
+              debugPrint('Error during Unicode support check:');
+              debugPrint('Error: $e');
+              debugPrint('Stack trace: $stackTrace');
+              rethrow;
+            }
+          },
+          onUpgrade: (db, oldVersion, newVersion) async {
+            debugPrint('Upgrading database from $oldVersion to $newVersion');
+
+            if (oldVersion < 2) {
+              // Check for Unicode support during upgrade
+              _hasIcuSupport = await _checkIcuSupport(db);
+              await _recreateFtsTable(db, _hasIcuSupport);
+            }
+          },
+        ),
+      );
+
+      // Store Unicode support status in settings for future reference
+      await setSetting('has_icu_support', _hasIcuSupport.toString());
+      debugPrint(
+        'Database opened and initialized (Unicode support: $_hasIcuSupport)',
+      );
+    } catch (e, stackTrace) {
+      debugPrint('Failed to initialize database:');
+      debugPrint('Error: $e');
+      debugPrint('Stack trace: $stackTrace');
       rethrow;
     }
   }
@@ -104,11 +345,73 @@ CREATE TABLE IF NOT EXISTS settings (
   }
 
   static Future<void> setSetting(String key, String value) async {
-    await db.insert(
+    await db.insert('settings', {
-      'settings',
+      'key': key,
-      {'key': key, 'value': value},
+      'value': value,
-      conflictAlgorithm: ConflictAlgorithm.replace,
+    }, conflictAlgorithm: ConflictAlgorithm.replace);
-    );
     debugPrint("Setting updated: $key = $value");
   }
+
+  // Search notes using FTS
+  static Future<List<Map<String, dynamic>>> searchNotes(String query) async {
+    try {
+      if (query.trim().isEmpty) {
+        return [];
+      }
+
+      // Split into individual terms, filter empty ones
+      List<String> terms =
+          query
+              .trim()
+              .split(RegExp(r'\s+'))
+              .where((term) => term.isNotEmpty)
+              .toList();
+
+      if (terms.isEmpty) {
+        return [];
+      }
+
+      // Process terms for FTS5 query using proper tokenization
+      String ftsQuery = terms
+          .map((term) {
+            // Remove dangerous characters but preserve Unicode
+            String sanitizedTerm = term.replaceAll(RegExp(r'''['"]'''), '');
+            if (sanitizedTerm.isEmpty) return '';
+
+            // Use proper FTS5 syntax: each word becomes a separate token with prefix matching
+            List<String> words = sanitizedTerm.split(RegExp(r'\s+'));
+            return words.map((word) => '$word*').join(' OR ');
+          })
+          .where((term) => term.isNotEmpty)
+          .join(' AND ');
+
+      if (ftsQuery.isEmpty) {
+        debugPrint('Query was sanitized to empty string');
+        return [];
+      }
+
+      debugPrint('FTS query: "$ftsQuery"');
+
+      // Execute the FTS query
+      final List<Map<String, dynamic>> results = await db.rawQuery(
+        '''
+        SELECT n.id, n.date, n.content,
+               snippet(notes_fts, -1, '<b>', '</b>', '...', 64) as snippet
+        FROM notes_fts
+        JOIN notes n ON notes_fts.rowid = n.id
+        WHERE notes_fts MATCH ?
+        ORDER BY rank
+        LIMIT 100
+        ''',
+        [ftsQuery],
+      );
+
+      debugPrint('Search returned ${results.length} results');
+      return results;
+    } catch (e, stackTrace) {
+      debugPrint('Search failed: $e');
+      debugPrint('Stack trace: $stackTrace');
+      return [];
+    }
+  }
 }

lib/notes.dart

@@ -1,10 +1,25 @@
 import 'package:journaler/db.dart';
+import 'package:intl/intl.dart';
 
 class Note {
   final String date;
+  late final String displayDate;
   String content;
+  String? snippet;
+  bool isProblematic;
+  String problemReason;
 
-  Note({required this.date, required this.content});
+  Note({
+    required this.date,
+    required this.content,
+    this.snippet,
+    this.isProblematic = false,
+    this.problemReason = '',
+  }) {
+    final dtUtc = DateFormat('yyyy-MM-dd HH:mm:ss').parse(date, true);
+    final dtLocal = dtUtc.toLocal();
+    displayDate = DateFormat('yyyy-MM-dd HH:mm:ss').format(dtLocal);
+  }
 }
 
 class Scratch {
@@ -28,16 +43,26 @@ Future<Note?> getLatestNote() async {
 }
 
 Future<void> createNote(String content) async {
-  if (content.isEmpty) {
+  // Trim the content to avoid saving just whitespace
+  final trimmedContent = content.trim();
+  if (trimmedContent.isEmpty) {
     return;
   }
-  await DB.db.insert('notes', {'content': content});
+  await DB.db.insert('notes', {'content': trimmedContent});
 }
 
 Future<void> updateNote(Note note) async {
+  // Trim the content to avoid saving just whitespace
+  final trimmedContent = note.content.trim();
+  if (trimmedContent.isEmpty) {
+    // Delete the note if content is empty
+    await DB.db.delete('notes', where: 'date = ?', whereArgs: [note.date]);
+    return;
+  }
+
   await DB.db.update(
     'notes',
-    {'content': note.content},
+    {'content': trimmedContent},
     where: 'date = ?',
     whereArgs: [note.date],
   );
@@ -59,7 +84,9 @@ Future<Scratch?> getLatestScratch() async {
 }
 
 Future<void> createScratch(String content) async {
-  await DB.db.insert('scratches', {'content': content});
+  // Trim content but allow empty scratch notes (they might be intentionally cleared)
+  final trimmedContent = content.trim();
+  await DB.db.insert('scratches', {'content': trimmedContent});
 }
 
 // Get the note immediately older than the given date
@@ -100,3 +127,75 @@ Future<Note?> getNextNote(String currentDate) async {
   // This handles the case where the `currentDate` might not be the absolute latest.
   return getLatestNote();
 }
+
+/// Delete a note by its date
+Future<bool> deleteNote(String date) async {
+  final result = await DB.db.delete(
+    'notes',
+    where: 'date = ?',
+    whereArgs: [date],
+  );
+
+  return result > 0; // Return true if a note was deleted
+}
+
+// Search notes using full-text search
+Future<List<Note>> searchNotes(String query) async {
+  if (query.isEmpty) {
+    return [];
+  }
+
+  // Call DB search function
+  final List<Map<String, dynamic>> results = await DB.searchNotes(query);
+
+  // Convert results to Note objects
+  return results
+      .map(
+        (result) => Note(
+          date: result['date'] as String,
+          content: result['content'] as String,
+          snippet:
+              result['snippet'] as String?, // Highlighted snippets from FTS
+        ),
+      )
+      .toList();
+}
+
+// Find potentially problematic entries based on character distribution
+Future<List<Note>> findProblematicEntries({
+  double maxCharPercentage = 0.7,
+}) async {
+  // Simple SQLite query that counts character occurrences using replace
+  final List<Map<String, dynamic>> results = await DB.db.rawQuery(
+    '''
+    WITH char_counts AS (
+      SELECT
+        id,
+        date,
+        content,
+        substr(content, 1, 1) as char,
+        (length(content) - length(replace(content, substr(content, 1, 1), ''))) as char_count,
+        length(content) as total_length,
+        cast(length(content) - length(replace(content, substr(content, 1, 1), '')) as float) / length(content) as percentage
+      FROM notes
+    )
+    SELECT *
+    FROM char_counts
+    WHERE percentage > ?
+    ORDER BY date DESC
+  ''',
+    [maxCharPercentage],
+  );
+
+  return results
+      .map(
+        (row) => Note(
+          date: row['date'] as String,
+          content: row['content'] as String,
+          isProblematic: true,
+          problemReason:
+              'Character "${row['char']}" makes up ${(row['percentage'] * 100).toStringAsFixed(1)}% of the content',
+        ),
+      )
+      .toList();
+}

pubspec.lock

@@ -224,6 +224,14 @@ packages:
       url: "https://pub.dev"
     source: hosted
     version: "4.5.4"
+  intl:
+    dependency: "direct main"
+    description:
+      name: intl
+      sha256: "3df61194eb431efc39c4ceba583b95633a403f46c9fd341e550ce0bfa50e9aa5"
+      url: "https://pub.dev"
+    source: hosted
+    version: "0.20.2"
   json_annotation:
     dependency: transitive
     description:
@@ -289,7 +297,7 @@ packages:
     source: hosted
     version: "1.16.0"
   path:
-    dependency: transitive
+    dependency: "direct main"
     description:
       name: path
       sha256: "75cca69d1490965be98c73ceaea117e8a04dd21217b37b292c9ddbec0d955bc5"
@@ -376,6 +384,14 @@ packages:
       url: "https://pub.dev"
     source: hosted
     version: "6.0.2"
+  ps_list:
+    dependency: "direct main"
+    description:
+      name: ps_list
+      sha256: "19d32f6c643313cf4f5101bb144b8978b9ba3dc42c9a01b247e8ed90581bc0ab"
+      url: "https://pub.dev"
+    source: hosted
+    version: "0.0.5"
   screen_retriever:
     dependency: transitive
     description:

pubspec.yaml

@@ -38,6 +38,9 @@ dependencies:
   window_manager: ^0.4.3
   audioplayers: ^6.4.0
   sqflite_common_ffi: ^2.3.5
+  path: ^1.8.0
+  ps_list: ^0.0.5
+  intl: ^0.20.2
 
 dev_dependencies:
   flutter_test:
@@ -66,6 +69,7 @@ flutter:
   assets:
     - assets/ # Include the main assets directory for the icon
     - assets/sounds/
+    - assets/windows/sqlite3_icu.dll
   #   - images/a_dot_burr.jpeg
   #   - images/a_dot_ham.jpeg