How We Train Our Models and Built Peachi OS

At COREA Starstroupe, a non-profit based in Cebu City, Philippines, we are dedicated to advancing open-source AI innovation. This page outlines the training methodologies for our Auralis and Nexora models and the development process for Peachi OS, as detailed in our research paper archive. Our work emphasizes lightweight, efficient, and interpretable AI solutions for resource-constrained environments and conversational intelligence.

Training Auralis: Lightweight and Interpretable NLP

Auralis is a compact natural language processing (NLP) model designed for instruction-following and real-time comprehension in low-resource settings. Our training approach, as documented in our papers, focuses on efficiency and interpretability.

Key Techniques

• Compact Architecture: As described in “Auralis v0.1: A Compact NLP Model” (July 2024), Auralis v0.1 uses a sub-million-parameter architecture optimized for microcontrollers. We employ a transformer-based design with reduced layers and attention heads to minimize computational overhead.
• Instruction-Following Fine-Tuning: We fine-tune Auralis on curated datasets of instruction-dialogue pairs to enhance its ability to extract intent and respond accurately. This process involves supervised learning with human-annotated examples, ensuring clarity in task execution.
• Interpretability Focus: The same paper highlights our use of attention visualization techniques to make Auralis’s decision-making transparent, allowing developers to understand how the model processes inputs.
• Dataset Optimization: In “Auralis: A Next-Generation Small Language Model” (January 2025), we discuss using synthetic data augmentation to expand training datasets while maintaining quality. This reduces reliance on large-scale data scraping and ensures ethical data practices.
• Resource-Constrained Training: We train Auralis on modest hardware, leveraging quantization techniques (e.g., 8-bit integer weights) to reduce memory usage, making the model deployable on edge devices.

Training Process

Our training pipeline for Auralis involves:

1. Data Preparation: Curating and cleaning datasets of dialogues and instructions, augmented with synthetic examples.
2. Pre-Training: Initial training on a general language corpus to establish base linguistic capabilities.
3. Fine-Tuning: Task-specific tuning for instruction-following and dialog intent extraction.
4. Optimization: Applying quantization and pruning to reduce model size without sacrificing performance.
5. Evaluation: Testing on benchmark tasks like intent recognition and response accuracy, with iterative refinement based on interpretability metrics.

These methods ensure Auralis is both powerful and practical for real-world applications, from smart assistants to educational tools.

Training Nexora: Compact NLP for Edge Devices

Nexora, including its NeuroLite variants, is designed for real-time NLP on resource-constrained devices like microcontrollers. Our training methodologies, detailed across multiple papers, prioritize low-latency inference and stability.

Key Techniques

• Sub-Million-Parameter Models: As outlined in “Nexora Proto v0.1” (November 2023), Nexora Proto uses a compact NLP stack with fewer than one million parameters, optimized for tokenization and embedding stability.
• Low-Latency Inference: In “NeuroLite-4M: A Compact Small Language Model” (December 2023), we describe training NeuroLite-4M with distilled datasets to enable real-time processing on edge devices, using techniques like knowledge distillation from larger models.
• Training Compression: “Nexora NeuroLite v0.6.2” (April 2024) details enhancements in training compression, such as weight pruning and sparse activations, to reduce computational requirements while maintaining accuracy.
• Instruction Tuning: In “Nexora Neurolite v1.0” (May 2024), we introduce advanced instruction tuning to improve task adaptability, using reinforcement learning with human feedback (RLHF) to align outputs with user expectations.
• Vector Integrity: We ensure stable embeddings through custom tokenization schemes, reducing drift during inference on low-power hardware.

Training Process

The Nexora training pipeline includes:

1. Dataset Distillation: Creating small, high-quality datasets from larger corpora to reduce training overhead.
2. Pre-Training: Building foundational language understanding on general text data.
3. Knowledge Distillation: Transferring knowledge from larger models to compact architectures.
4. Fine-Tuning with RLHF: Aligning the model with specific NLP tasks using human feedback loops.
5. Optimization and Deployment: Applying compression techniques and testing on microcontrollers to ensure low-latency performance.

This approach allows Nexora to power NLP applications in environments with limited computational resources, such as IoT devices and embedded systems.

Building Peachi OS: A Lightweight AI Operating System

Peachi OS is a lightweight operating system designed for NLP workloads, emphasizing modularity and conversational intelligence. Our development process, as documented in our papers, reflects a commitment to open-source innovation.

Key Development Principles

• Microkernel Modularity: As detailed in “PEACHI OS: Foundational Implementation” (March 2024), Peachi OS uses a microkernel architecture to ensure flexibility and scalability. Core components, like token scheduling and memory management, are isolated for stability.
• Intent Modeling: In “Intent Vector System” (August 2024), we introduce a computational framework for real-time user intent modeling, enabling Peachi OS to interpret and respond to user commands dynamically.
• Adaptive Kernel: “PEACHI OS: Adaptive Kernel Computation” (October 2024) describes advancements in dynamic computation, allowing the kernel to adjust resource allocation based on conversational demands.
• Lightweight Design: Peachi OS is optimized for low-resource environments, using minimal system calls and efficient data structures to support NLP tasks.
• Open-Source Collaboration: We develop Peachi OS transparently, releasing code and documentation under open-source licenses to foster community contributions.

Development Process

The creation of Peachi OS involved:

1. Architecture Design: Defining a microkernel structure with modular components for NLP processing, memory management, and user interaction.
2. Core Implementation: Building foundational systems, such as token scheduling and intent vector processing, using C and Assembly for performance.
3. Integration of NLP Models: Embedding models like Auralis and Nexora to handle conversational tasks, with optimized interfaces for real-time dialogue.
4. Testing and Refinement: Conducting rigorous testing on low-power hardware to ensure stability and efficiency, with iterative improvements based on performance metrics.
5. Documentation and Release: Publishing detailed documentation and source code to engage the open-source community and support further development.

Peachi OS powers conversational AI applications, from chatbots to voice assistants, with a focus on accessibility and adaptability.

Our Commitment to Open-Source AI

At COREA Starstroupe, we believe in transparent, ethical, and accessible AI development. Our training and development processes for Auralis, Nexora, and Peachi OS reflect this mission, prioritizing efficiency, interpretability, and community collaboration. For more technical details, explore our research paper archive, where we share our methodologies and findings openly.

If you have questions or wish to contribute, contact us at contact@coreastarstroupe.org. Join us in advancing AI for the global good!