AI Security Lab

Specific inquiry details. I was introduced to AutoML, which is said to make it easy to build machine learning models even without deep expert knowledge of machine learning, as long as you have a strong understanding of the business domain.Among the AutoML tools, I chose PyCaret because it is considered stable and integrates well with […]

Specific inquiry details. I work in the information security team at my company and am in charge of security reviews.Our company is planning to build a generative AI (GenAI) system.Since building a GenAI system differs from building a conventional information system, I’m wondering how to approach it.Also, please tell me what risks generative AI poses.

OWASP Top 10 for LLM Application LLM01. Prompt InjectionMalicious users may manipulate the LLM (GenAI) to redefine system prompts or induce unintended actions through external inputs, leading to data leakage or social engineering attacks. LLM02. Insecure Output HandlingIf the outputs generated by the LLM (GenAI) are not properly validated, vulnerabilities such as XSS, CSRF, or

Overview of RAG RAG (Retrieval-Augmented Generation) has become an essential component, alongside PEFT, in the development of GenAI (Large Language Model) systems. It is a technology designed to overcome the inherent limitations of standard LLM architectures. RAG enhances a model’s ability to provide accurate and contextually relevant answers by retrieving and referencing external documents during

Recently, many domestic financial companies have been building GenAI (Generative AI, LLM) systems. However, discussions on LLM security vulnerabilities have been limited to the risk of personal information leakage, while various information security considerations defined in the OWASP Top 10 for LLM Applications have not been taken into account. 1. Vulnerability Test This vulnerability maps

Prompt Injection, one of the key vulnerabilities in GenAI (LLM) systems, appears as the first item in the OWASP Top 10 for LLM Applications due to the wide range of existing attack techniques. At the same time, modern LLM-based services inevitably rely on multiple types of prompts to deliver accurate instructions and consistent system behavior.

In preparation for our GenAI (LLM) Red Team service, we organized a comprehensive set of detailed evaluation test cases aligned with the OWASP Top 10 for LLMs.

Name Full Name Architecture Base Model Developed Training Dataset Lib. & Framework Use Cases HF URL Githhub URL TimeSformer TimeSformer (Time-Space Transformer) Transformer Vision Transformer (ViT) 2021 Evaluated on datasets like Kinetics-400 and Kinetics-600 PyTorch Video classification and action recognition tasks https://github.com/facebookresearch/TimeSformer VideoMAE Video Masked Autoencoders Masked autoencoder Vision Transformer (ViT) 2022 Pre-trained on large-scale

Name Full Name Architecture Base Model Developed Training Dataset Lib. & Framework Use Cases HF URL Githhub URL Audio Spectrogram Transformer Audio Spectrogram Transformer Transformer ViT 2021 AudioSet PyTorch, Hugging Face Transformers Audio classification, sound event detection https://huggingface.co/docs/transformers/model_doc/audio-spectrogram-transformer https://github.com/YuanGongND/ast Bark Bark GPT-like, Transformer GPT-2 2023 Proprietary dataset PyTorch, Hugging Face Transformers Text-to-speech, voice synthesis https://huggingface.co/docs/transformers/model_doc/bark

Name Full Name Architecture Base Model Developed Training Dataset Lib. & Framework Use Cases HF URL Githhub URL BEiT Bidirectional Encoder representation from Image Transformers Vision Transformer ViT 2021 ImageNet-21k, ImageNet-1k PyTorch, Hugging Face Transformers Image classification, semantic segmentation https://huggingface.co/microsoft/beit-base-patch16-224 https://github.com/microsoft/unilm/tree/master/beit BiT Big Transfer ResNet ResNet 2019 JFT-300M, ImageNet-21k TensorFlow, Hugging Face Transformers Image classification, transfer learning