ෆ♡ෆ

ʚ(｡˃ ᵕ ˂ )ɞ

ෆ♡ෆ ʚ(｡˃ ᵕ ˂ )ɞ

✰𝑨𝑰 𝑺𝑪𝑯𝑬𝑴𝑨 𝑭𝑶𝑹 𝑬𝑼𝑺𝑶𝑵𝑼𝑺✰

🪷 Level 1 🪷

Sound 🎤 Recognition 🎼 Harmonization

⭐Objective⭐

⭐ Develop AI algorithms for sound recognition and basic harmonization.

🦋Tasks🦋

🐛 Implement and test microphone access for various VR devices.

🐛 Develop frequency detection and sound classification algorithms.

🐛 Define musical scales and rules for harmonization.

🐛 Develop real-time harmonization algorithms and enable user customization.

Level 1: Sound Recognition and Harmonization

1.1 Microphone Integration and Sound Recognition

  • Task: Ensure the VR headset or device can access and capture live audio.

  • Roadmap:

    • Select compatible microphone access library/API, implement and test across VR devices.

1.2 Basic Harmonization

  • Task: Harmonize recognized sound based on predefined scales and rules.

  • Roadmap:

    • Research existing frequency detection algorithms. Some potential references from GitHub include:

      • DeepSpeech: An open-source speech-to-text engine.

      • Whisper: OpenAI's model for speech recognition.

      • Leon: An open-source personal assistant with sound recognition capabilities.

    • Integrate the chosen algorithm into the experience.

    • Define scales and rules, implement harmonization algorithm, and enable user customization.

    • Test the algorithm's accuracy with various sound inputs.

1.3 Sound Classification

  • Task: Classify the detected sound into predefined categories or moods.

  • Roadmap:

    • Define the categories or moods you want to classify the sounds into.

    • Train a machine learning model using labeled sound data or use a pre-trained model.

    • Integrate the classification model into the experience.

    • Test the model's accuracy and refine as necessary.

1.4 Real-time Processing

  • Task: Ensure the sound recognition and processing happen in real-time without noticeable lag.

  • Roadmap:

    • Optimize the algorithms for speed and efficiency.

    • Test the experience in real-world scenarios to ensure real-time processing.

    • Make necessary adjustments based on feedback and testing results.

1.5 Feedback Loop

  • Task: Implement a feedback mechanism for users to correct any misclassifications or provide input on the recognized sound.

  • Roadmap:

    • Design a user-friendly interface for feedback within the experience.

    • Use the feedback to continuously improve the sound recognition algorithm.

    • Periodically retrain the model with new data and feedback.