diff --git a/README.md b/README.md index 071392ed7..c0365ed9b 100644 --- a/README.md +++ b/README.md @@ -587,7 +587,7 @@ Example: ![](readme-assets/shortcode-snippets.png) This shortcode renders a code snippets widget from a specified path. -Use it when you want to manage code examples as a collection of separate Markdown files. +Use it when you want to manage code examples as a collection of separate Markdown files. The following parapghs refer to hand-written snippets. It's recommended to write code snippets as testable code instead. Refer to [automation/snippets/README.md](automation/snippets/README.md) for details. ##### 📁 Directory Structure Place all code snippets for a single widget into one directory. Each file should be named after the programming language it represents: diff --git a/automation/snippets/README.md b/automation/snippets/README.md index 23bdf67c1..bbff791b9 100644 --- a/automation/snippets/README.md +++ b/automation/snippets/README.md @@ -8,6 +8,89 @@ This is a tool to work with code snippets. Also, it supports testing snippets against unreleased versions of clients (from git branches/PRs). +## TL;DR + +```bash +cd automation/snippets + +# Fetch the latest clients +./docker.sh ./sync-clients.py fetch --csharp --java --typescript + +# Test whether the snippets compile +./docker.sh ./check.py build snippets/path/to/snippets/* + +# Generate the Markdown files +./docker.sh ./generate-md.py +``` + +## Usage + +Use this tooling to write snippets as runnable code, test their validity, and generate Markdown files from them. + +Place your snippets as code files (for example, `python.py`) in the `qdrant-landing/content/documentation/headless/snippets/` directory. You can organize them in subdirectories. Some languages (like Java) require boilerplate code. This boilerplate code is automatically hidden in the generated Markdown files. Refer to existing snippets for examples. + +After running `generate-md.py`, the generated Markdown files will be placed in the `generated/` subdirectory next to the code files. You can then include these generated Markdown files in your documentation using the `code-snippet` shortcode: + +``` +{{< code-snippet path="/documentation/headless/snippets/example/" >}} +``` + +Your code may need some boilerplate code that you don't want to show in the generated Markdown files. You can hide such code by placing `// @hide` (or `# @hide` for Python) at the end of the line. Entire blocks of code can be hidden by placing `// @hide-start` and `// @hide-end` around the block. + +You can generate multiple code snippets from one source file by defining "blocks" inside the code. This is useful for tutorials, where later code depends on classes and variables defined in earlier code. Each block becomes its own Markdown snippet, and a Markdown snippet is also generated for the entire file. Use `// @block-start block-name` and `// @block-end block-name` to define a block. + +Include a block in your documentation using the `code-snippet` shortcode with the `block` parameter: + +``` +{{< code-snippet path="/documentation/headless/snippets/example/" block="block-name" >}} +``` + +# Example + +The following Python code: + +```python +some_boilerplate_initialization_code() # @hide + +print("Hello") + +# @block-start world +print("World") +# @block-end world + +# @hide-start +some_boilerplate_cleanup_code() +# @hide-end +``` + +Results in the following directory structure: + +```. +├── generated +│ └── python.md +│ └── world +│ └── python.md +├── python.py +``` + +with `generated/python.md` containing: + +```` +```python +print("Hello") + +print("World") +``` +```` + +and `generated/world/python.md` containing: + +```` +```python +print("World") +``` +```` + ## Dependencies To convert runnable snippets into markdown ([`generate-md.py`](./generate-md.py)) you need only python with no extra dependencies. diff --git a/automation/snippets/generate-md.py b/automation/snippets/generate-md.py index 5c77ad231..c5f7be5f5 100755 --- a/automation/snippets/generate-md.py +++ b/automation/snippets/generate-md.py @@ -13,6 +13,7 @@ The reverse is `./migrate-snippet.py`. import difflib import shutil import sys +import textwrap import traceback import typing @@ -39,29 +40,33 @@ def main() -> None: print(f"Warning: failed to shorten snippet {snippet_fname}: {e}") traceback.print_exc() continue - generated = f"```{lang.NAME}\n{shortened}```\n" - generated_fname = generated_dir / f"{lang.NAME}.md" - handwritten_fname = snippet_dir / f"{lang.NAME}.md" - generated_fname.write_text(generated) + for key in shortened.keys(): + generated = f"```{lang.NAME}\n{textwrap.dedent(shortened[key])}```\n" - if handwritten_fname.exists(): - issues += 1 - print( - "Warning: both snippet and generated file exist:", - snippet_dir / f"{lang.NAME}.md", - ) - handwritten = (snippet_dir / f"{lang.NAME}.md").read_text() - if handwritten.rstrip("\n") != generated.rstrip("\n"): - print_and_colorize_diff( - difflib.unified_diff( - handwritten.rstrip("\n").splitlines(keepends=True), - generated.rstrip("\n").splitlines(keepends=True), - fromfile=str(handwritten_fname), - tofile=str(generated_fname), - ), + generated_dir = snippet_dir / "generated" / key + generated_dir.mkdir(exist_ok=True) + generated_fname = generated_dir / f"{lang.NAME}.md" + handwritten_fname = snippet_dir / f"{lang.NAME}.md" + generated_fname.write_text(generated) + + if handwritten_fname.exists(): + issues += 1 + print( + "Warning: both snippet and generated file exist:", + snippet_dir / f"{lang.NAME}.md", ) - print() + handwritten = (snippet_dir / f"{lang.NAME}.md").read_text() + if handwritten.rstrip("\n") != generated.rstrip("\n"): + print_and_colorize_diff( + difflib.unified_diff( + handwritten.rstrip("\n").splitlines(keepends=True), + generated.rstrip("\n").splitlines(keepends=True), + fromfile=str(handwritten_fname), + tofile=str(generated_fname), + ), + ) + print() if issues: print(f"Total issues found: {issues}") diff --git a/automation/snippets/lib/languages/base.py b/automation/snippets/lib/languages/base.py index 48b2ee9c8..a174b3aed 100644 --- a/automation/snippets/lib/languages/base.py +++ b/automation/snippets/lib/languages/base.py @@ -26,11 +26,12 @@ class Language: raise NotImplementedError @classmethod - def shorten(cls, contents: str) -> str: + def shorten(cls, contents: str) -> dict[str, str]: """Shorten the snippet contents into a form suitable for inclusion in - documentation. + documentation. Also splits the snippet into blocks. Removes boilerplate code, e.g. class wrappers, main functions, etc. + Returns mapping `block_name` -> `block_contents`. """ return generic_shorten(contents) @@ -81,51 +82,94 @@ def template( target_fname.write_text("".join(result)) -_RE_COMMENT = re.compile(r"^(.*\s|)(?://|#)\s*(@.*)$") +_RE_COMMENT = re.compile( + r""" + ^ + (?P .*\s | ) # code before comment + (?: // | \# ) # comment start + \s* + (?P @\S+ ) + (?P \s+ .* )? + $ + """, + re.VERBOSE, +) -def generic_shorten(text: str) -> str: +def generic_shorten(text: str) -> dict[str, str]: """Generic implementation of Language.shorten(). - Removes comments with @hide annotation and trims excessive newlines. + Processes annotation comments (@hide, @block-start, etc.), trims excessive + newlines, and splits into blocks. """ - result = [] + blocks = { + # empty string is the default block (does not live in a subdirectory) + "": [] + } + current_blocks = [""] hide_mode = False for line in text.splitlines(): if (m := _RE_COMMENT.match(line)) is None: if not hide_mode: - result.append(line + "\n") + for block in current_blocks: + blocks[block].append(line + "\n") continue - has_code = m[1].strip() != "" - annotation = m[2] + has_code = m["code"].strip() != "" + annotation = m["annotation"] + param = m["param"].strip() if m["param"] is not None else "" if annotation == "@hide": if not has_code: raise ValueError("Hiding empty line is not allowed") if hide_mode: raise ValueError("@hide inside @hide-start/@hide-end is not allowed") + if param: + raise ValueError("@hide should not be followed by any parameters") elif annotation == "@hide-start": if has_code: raise ValueError("@hide-start should be on its own line") if hide_mode: raise ValueError("Nesting @hide-start is not allowed") + if param: + raise ValueError("@hide-start should not be followed by any parameters") hide_mode = True elif annotation == "@hide-end": if has_code: raise ValueError("@hide-end should be on its own line") if not hide_mode: raise ValueError("@hide-end without matching @hide-start") + if param: + raise ValueError("@hide-end should not be followed by any parameters") hide_mode = False + elif annotation == "@block-start": + if has_code: + raise ValueError("@block-start should be on its own line") + if param: + current_blocks.append(param) + blocks[param] = [] + else: + raise ValueError("@block-start should be followed by block name") + elif annotation == "@block-end": + if has_code: + raise ValueError("@block-end should be on its own line") + if param: + current_blocks.remove(param) + else: + raise ValueError("@block-end should be followed by block name") else: - raise ValueError(f"Unknown annotation: {m[1]}") + raise ValueError(f"Unknown annotation: {annotation}") if hide_mode: raise ValueError("Unclosed @hide-start") - text = "".join(result) - text = text.lstrip("\n").rstrip("\n") - text = re.sub(r"\n{3,}", "\n\n", text) - if text: - text += "\n" - return text + + snippets = {} + for key in blocks.keys(): + text = "".join(blocks[key]) + text = text.lstrip("\n").rstrip("\n") + text = re.sub(r"\n{3,}", "\n\n", text) + if text: + text += "\n" + snippets[key] = text + return snippets def trim_commonpath(fnames: list[Path]) -> dict[Path, Path]: diff --git a/automation/snippets/lib/languages/csharp.py b/automation/snippets/lib/languages/csharp.py index 23f464fd6..31569efc5 100644 --- a/automation/snippets/lib/languages/csharp.py +++ b/automation/snippets/lib/languages/csharp.py @@ -86,7 +86,7 @@ class LanguageCsharp(Language): assert RE_CODE.match(EXAMPLE_CODE) is not None @classmethod - def shorten(cls, contents: str) -> str: + def shorten(cls, contents: str) -> dict[str, str]: if (m := LanguageCsharp.RE_CODE.match(contents)) is None: msg = "Invalid snippet format" raise ValueError(msg) diff --git a/automation/snippets/lib/languages/go.py b/automation/snippets/lib/languages/go.py index 28f4cd423..7e99cc2c9 100644 --- a/automation/snippets/lib/languages/go.py +++ b/automation/snippets/lib/languages/go.py @@ -82,7 +82,7 @@ class LanguageGo(Language): assert RE_CODE.match(EXAMPLE_CODE) is not None @classmethod - def shorten(cls, contents: str) -> str: + def shorten(cls, contents: str) -> dict[str, str]: if (m := LanguageGo.RE_CODE.match(contents)) is None: msg = "Invalid snippet format" raise ValueError(msg) diff --git a/automation/snippets/lib/languages/java.py b/automation/snippets/lib/languages/java.py index 5494bdebb..58d228f5b 100644 --- a/automation/snippets/lib/languages/java.py +++ b/automation/snippets/lib/languages/java.py @@ -94,7 +94,7 @@ class LanguageJava(Language): assert RE_CODE.match(EXAMPLE_CODE) is not None @classmethod - def shorten(cls, contents: str) -> str: + def shorten(cls, contents: str) -> dict[str, str]: if (m := LanguageJava.RE_CODE.match(contents)) is None: msg = "Invalid snippet format" raise ValueError(msg) diff --git a/automation/snippets/lib/languages/python.py b/automation/snippets/lib/languages/python.py index 6e663ec55..5c7ebb12b 100644 --- a/automation/snippets/lib/languages/python.py +++ b/automation/snippets/lib/languages/python.py @@ -55,7 +55,7 @@ class LanguagePython(Language): return result @classmethod - def shorten(cls, contents: str) -> str: + def shorten(cls, contents: str) -> dict[str, str]: lines = [ line for line in contents.splitlines(keepends=True) diff --git a/automation/snippets/lib/languages/rust.py b/automation/snippets/lib/languages/rust.py index 94b33f266..c24d7dbe3 100644 --- a/automation/snippets/lib/languages/rust.py +++ b/automation/snippets/lib/languages/rust.py @@ -91,7 +91,7 @@ class LanguageRust(Language): assert RE_CODE.match(EXAMPLE_CODE) is not None @classmethod - def shorten(cls, contents: str) -> str: + def shorten(cls, contents: str) -> dict[str, str]: if (m := LanguageRust.RE_CODE.match(contents)) is None: msg = "Invalid snippet format" raise ValueError(msg) diff --git a/qdrant-landing/content/documentation/headless/snippets/cloud-inference/vector-search/upload-data/generated/python.md b/qdrant-landing/content/documentation/headless/snippets/cloud-inference/vector-search/upload-data/generated/python.md index 4c802e3db..78d5d5439 100644 --- a/qdrant-landing/content/documentation/headless/snippets/cloud-inference/vector-search/upload-data/generated/python.md +++ b/qdrant-landing/content/documentation/headless/snippets/cloud-inference/vector-search/upload-data/generated/python.md @@ -13,7 +13,7 @@ points = [] for idx, item in enumerate(ds): passage = item["passage_text"] - + point = PointStruct( id=uuid.uuid4().hex, # use unique string ID payload=item, diff --git a/qdrant-landing/content/documentation/headless/snippets/cloud-inference/vector-search/upload-data/python.py b/qdrant-landing/content/documentation/headless/snippets/cloud-inference/vector-search/upload-data/python.py index e07afde1d..f698a11e2 100644 --- a/qdrant-landing/content/documentation/headless/snippets/cloud-inference/vector-search/upload-data/python.py +++ b/qdrant-landing/content/documentation/headless/snippets/cloud-inference/vector-search/upload-data/python.py @@ -16,7 +16,7 @@ points = [] for idx, item in enumerate(ds): passage = item["passage_text"] - + point = PointStruct( id=uuid.uuid4().hex, # use unique string ID payload=item, diff --git a/qdrant-landing/content/documentation/headless/snippets/install-client/csharp.md b/qdrant-landing/content/documentation/headless/snippets/install-client/csharp.md new file mode 100644 index 000000000..73022fdf7 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/install-client/csharp.md @@ -0,0 +1,3 @@ +```csharp +Qdrant.Client +``` \ No newline at end of file diff --git a/qdrant-landing/content/documentation/headless/snippets/install-client/go.md b/qdrant-landing/content/documentation/headless/snippets/install-client/go.md new file mode 100644 index 000000000..70db873e4 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/install-client/go.md @@ -0,0 +1,3 @@ +```go +github.com/qdrant/go-client +``` \ No newline at end of file diff --git a/qdrant-landing/content/documentation/headless/snippets/install-client/java.md b/qdrant-landing/content/documentation/headless/snippets/install-client/java.md new file mode 100644 index 000000000..d5752540a --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/install-client/java.md @@ -0,0 +1,3 @@ +```java +io.qdrant:client +``` \ No newline at end of file diff --git a/qdrant-landing/content/documentation/headless/snippets/install-client/python.md b/qdrant-landing/content/documentation/headless/snippets/install-client/python.md new file mode 100644 index 000000000..584b7ef93 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/install-client/python.md @@ -0,0 +1,3 @@ +```python +qdrant-client +``` \ No newline at end of file diff --git a/qdrant-landing/content/documentation/headless/snippets/install-client/rust.md b/qdrant-landing/content/documentation/headless/snippets/install-client/rust.md new file mode 100644 index 000000000..b22bc111c --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/install-client/rust.md @@ -0,0 +1,3 @@ +```rust +qdrant-client +``` \ No newline at end of file diff --git a/qdrant-landing/content/documentation/headless/snippets/install-client/typescript.md b/qdrant-landing/content/documentation/headless/snippets/install-client/typescript.md new file mode 100644 index 000000000..c98ea2c67 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/install-client/typescript.md @@ -0,0 +1,3 @@ +```typescript +qdrant/js-client-rest +``` \ No newline at end of file diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/_description.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/_description.md new file mode 100644 index 000000000..9695ce38b --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/_description.md @@ -0,0 +1 @@ +This code snippet shows how to create a semantic search engine with Qdrant. First, a client connection to Qdrant Cloud is established using the cluster URL and API key, and cloud inference is enabled for automatic embedding generation. Next, a collection named `my_books` is created with a vector size of 384 and cosine distance. A dataset of science fiction books is defined, and each book is stored as a point in the collection with a unique ID, a vector generated from the description, and a payload containing the book's metadata. Finally, a query is made to the search engine to find books related to an alien invasion, and the most relevant results are returned with their similarity scores. The code also demonstrates how to apply a filter to query in order to return only books published after the year 2000. diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/client-connection/_description.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/client-connection/_description.md deleted file mode 100644 index 7193da14d..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/client-connection/_description.md +++ /dev/null @@ -1 +0,0 @@ -This code snippet shows how to create a client connection to Qdrant Cloud, with the cluster URL and API key, and enables cloud inference for automatic embedding generation. diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/client-connection/python.py b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/client-connection/python.py deleted file mode 100644 index 39794e26a..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/client-connection/python.py +++ /dev/null @@ -1,12 +0,0 @@ -# @hide-start -QDRANT_URL="" -QDRANT_API_KEY="" -# @hide-end - -from qdrant_client import QdrantClient, models - -client = QdrantClient( - url=QDRANT_URL, - api_key=QDRANT_API_KEY, - cloud_inference=True -) \ No newline at end of file diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/_description.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/_description.md deleted file mode 100644 index 02d56563d..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/_description.md +++ /dev/null @@ -1 +0,0 @@ -This code snippet shows how to create a collection in Qdrant. The example creates a collection named `my_books` configured to store 384-dimensional vectors with cosine distance metric. diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/python.py b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/python.py deleted file mode 100644 index 1fb539c67..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/python.py +++ /dev/null @@ -1,19 +0,0 @@ -# @hide-start -from qdrant_client import QdrantClient, models - -client = QdrantClient( - url="", - api_key="", - cloud_inference=True -) -# @hide-end - -COLLECTION_NAME="my_books" - -client.create_collection( - collection_name=COLLECTION_NAME, - vectors_config=models.VectorParams( - size=384, # Vector size is defined by the model - distance=models.Distance.COSINE, - ), -) \ No newline at end of file diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-payload-index/_description.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-payload-index/_description.md deleted file mode 100644 index fdc26ac5b..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-payload-index/_description.md +++ /dev/null @@ -1 +0,0 @@ -This code snippet shows how to create a payload index on a specific field. The example creates an index on the `year` field of type integer, which enables efficient filtering on this field in subsequent queries. diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-payload-index/python.py b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-payload-index/python.py deleted file mode 100644 index 104dc2a88..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-payload-index/python.py +++ /dev/null @@ -1,17 +0,0 @@ -# @hide-start -from qdrant_client import QdrantClient, models - -client = QdrantClient( - url="", - api_key="", - cloud_inference=True -) - -COLLECTION_NAME="my_books" -# @hide-end - -client.create_payload_index( - collection_name=COLLECTION_NAME, - field_name="year", - field_schema=models.PayloadSchemaType.INTEGER, -) diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/csharp.cs b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/csharp.cs new file mode 100644 index 000000000..49b801b41 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/csharp.cs @@ -0,0 +1,126 @@ +using Qdrant.Client; +using Qdrant.Client.Grpc; +using static Qdrant.Client.Grpc.Conditions; + +public class Snippet +{ + public static async Task Run() + { + // @hide-start + string QDRANT_URL = ""; + string QDRANT_API_KEY = ""; + // @hide-end + // @block-start client-connection + var client = new QdrantClient( + host: QDRANT_URL, + port: 6334, + https: true, + apiKey: QDRANT_API_KEY + ); + // @block-end client-connection + + // @block-start create-collection + string COLLECTION_NAME = "my_books"; + + await client.CreateCollectionAsync( + collectionName: COLLECTION_NAME, + vectorsConfig: new VectorParams { Size = 384, Distance = Distance.Cosine } + ); + // @block-end create-collection + + // @block-start upload-data + var payloads = new List> + { + new() { ["name"] = "The Time Machine", ["description"] = "A man travels through time and witnesses the evolution of humanity.", ["author"] = "H.G. Wells", ["year"] = 1895 }, + new() { ["name"] = "Ender's Game", ["description"] = "A young boy is trained to become a military leader in a war against an alien race.", ["author"] = "Orson Scott Card", ["year"] = 1985 }, + new() { ["name"] = "Brave New World", ["description"] = "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", ["author"] = "Aldous Huxley", ["year"] = 1932 }, + new() { ["name"] = "The Hitchhiker's Guide to the Galaxy", ["description"] = "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", ["author"] = "Douglas Adams", ["year"] = 1979 }, + new() { ["name"] = "Dune", ["description"] = "A desert planet is the site of political intrigue and power struggles.", ["author"] = "Frank Herbert", ["year"] = 1965 }, + new() { ["name"] = "Foundation", ["description"] = "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", ["author"] = "Isaac Asimov", ["year"] = 1951 }, + new() { ["name"] = "Snow Crash", ["description"] = "A futuristic world where the internet has evolved into a virtual reality metaverse.", ["author"] = "Neal Stephenson", ["year"] = 1992 }, + new() { ["name"] = "Neuromancer", ["description"] = "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", ["author"] = "William Gibson", ["year"] = 1984 }, + new() { ["name"] = "The War of the Worlds", ["description"] = "A Martian invasion of Earth throws humanity into chaos.", ["author"] = "H.G. Wells", ["year"] = 1898 }, + new() { ["name"] = "The Hunger Games", ["description"] = "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", ["author"] = "Suzanne Collins", ["year"] = 2008 }, + new() { ["name"] = "The Andromeda Strain", ["description"] = "A deadly virus from outer space threatens to wipe out humanity.", ["author"] = "Michael Crichton", ["year"] = 1969 }, + new() { ["name"] = "The Left Hand of Darkness", ["description"] = "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", ["author"] = "Ursula K. Le Guin", ["year"] = 1969 }, + new() { ["name"] = "The Three-Body Problem", ["description"] = "Humans encounter an alien civilization that lives in a dying system.", ["author"] = "Liu Cixin", ["year"] = 2008 } + }; + // @block-end upload-data + + // @block-start upload-points + string EMBEDDING_MODEL = "sentence-transformers/all-minilm-l6-v2"; + + var points = new List(); + + for (ulong idx = 0; idx < (ulong)payloads.Count; idx++) + { + var payload = payloads[(int)idx]; + string description = payload["description"].StringValue; + + var point = new PointStruct + { + Id = idx, + Vectors = new Document + { + Text = description, + Model = EMBEDDING_MODEL + }, + Payload = { payload } + }; + + points.Add(point); + } + + await client.UpsertAsync( + collectionName: COLLECTION_NAME, + points: points + ); + // @block-end upload-points + + // @block-start query-engine + var hits = await client.QueryAsync( + collectionName: COLLECTION_NAME, + query: new Document + { + Text = "alien invasion", + Model = EMBEDDING_MODEL + }, + limit: 3 + ); + + foreach (var hit in hits) + { + Console.WriteLine($"{hit.Payload} score: {hit.Score}"); + } + // @block-end query-engine + + // @block-start create-payload-index + await client.CreatePayloadIndexAsync( + collectionName: COLLECTION_NAME, + fieldName: "year", + schemaType: PayloadSchemaType.Integer + ); + // @block-end create-payload-index + + // @block-start query-with-filter + var filteredHits = await client.QueryAsync( + collectionName: COLLECTION_NAME, + query: new Document + { + Text = "alien invasion", + Model = EMBEDDING_MODEL + }, + filter: new Filter + { + Must = { Range("year", new Qdrant.Client.Grpc.Range { Gte = 2000.0 }) } + }, + limit: 1 + ); + + foreach (var hit in filteredHits) + { + Console.WriteLine($"{hit.Payload} score: {hit.Score}"); + } + // @block-end query-with-filter + } +} diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/csharp.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/csharp.md new file mode 100644 index 000000000..319930449 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/csharp.md @@ -0,0 +1,8 @@ +```csharp +var client = new QdrantClient( + host: QDRANT_URL, + port: 6334, + https: true, + apiKey: QDRANT_API_KEY +); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/go.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/go.md new file mode 100644 index 000000000..c00c087d3 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/go.md @@ -0,0 +1,7 @@ +```go +client, err := qdrant.NewClient(&qdrant.Config{ + Host: QDRANT_URL, + APIKey: QDRANT_API_KEY, + UseTLS: true, +}) +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/java.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/java.md new file mode 100644 index 000000000..5b070ed18 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/java.md @@ -0,0 +1,7 @@ +```java +QdrantClient client = + new QdrantClient( + QdrantGrpcClient.newBuilder(QDRANT_URL, 6334, true) + .withApiKey(QDRANT_API_KEY) + .build()); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/client-connection/generated/python.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/python.md similarity index 100% rename from qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/client-connection/generated/python.md rename to qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/python.md diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/rust.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/rust.md new file mode 100644 index 000000000..8862eafb1 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/rust.md @@ -0,0 +1,5 @@ +```rust +let client = Qdrant::from_url(QDRANT_URL) + .api_key(QDRANT_API_KEY) + .build()?; +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/typescript.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/typescript.md new file mode 100644 index 000000000..27af73240 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/client-connection/typescript.md @@ -0,0 +1,6 @@ +```typescript +const client = new QdrantClient({ + url: QDRANT_URL, + apiKey: QDRANT_API_KEY, +}); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/csharp.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/csharp.md new file mode 100644 index 000000000..4febc36e5 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/csharp.md @@ -0,0 +1,8 @@ +```csharp +string COLLECTION_NAME = "my_books"; + +await client.CreateCollectionAsync( + collectionName: COLLECTION_NAME, + vectorsConfig: new VectorParams { Size = 384, Distance = Distance.Cosine } +); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/go.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/go.md new file mode 100644 index 000000000..d8d4d9837 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/go.md @@ -0,0 +1,11 @@ +```go +collectionName := "my_books" + +client.CreateCollection(context.Background(), &qdrant.CreateCollection{ + CollectionName: collectionName, + VectorsConfig: qdrant.NewVectorsConfig(&qdrant.VectorParams{ + Size: 384, // Vector size is defined by used model + Distance: qdrant.Distance_Cosine, + }), +}) +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/java.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/java.md new file mode 100644 index 000000000..1ed944e66 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/java.md @@ -0,0 +1,6 @@ +```java +String COLLECTION_NAME = "my_books"; + +client.createCollectionAsync(COLLECTION_NAME, + VectorParams.newBuilder().setDistance(Distance.Cosine).setSize(384).build()).get(); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/generated/python.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/python.md similarity index 77% rename from qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/generated/python.md rename to qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/python.md index b86220f63..26585437d 100644 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/generated/python.md +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/python.md @@ -4,7 +4,7 @@ COLLECTION_NAME="my_books" client.create_collection( collection_name=COLLECTION_NAME, vectors_config=models.VectorParams( - size=384, # Vector size is defined by the model + size=384, # Vector size is defined by used model distance=models.Distance.COSINE, ), ) diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/rust.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/rust.md new file mode 100644 index 000000000..7783e603f --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/rust.md @@ -0,0 +1,10 @@ +```rust +let collection_name = "my_books"; + +client + .create_collection( + CreateCollectionBuilder::new(collection_name) + .vectors_config(VectorParamsBuilder::new(384, Distance::Cosine)), // Vector size is defined by used model + ) + .await?; +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/typescript.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/typescript.md new file mode 100644 index 000000000..273070490 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-collection/typescript.md @@ -0,0 +1,10 @@ +```typescript +const collectionName = "my_books"; + +await client.createCollection(collectionName, { + vectors: { + size: 384, // Vector size is defined by used model + distance: "Cosine", + }, +}); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/csharp.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/csharp.md new file mode 100644 index 000000000..b16b435ba --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/csharp.md @@ -0,0 +1,7 @@ +```csharp +await client.CreatePayloadIndexAsync( + collectionName: COLLECTION_NAME, + fieldName: "year", + schemaType: PayloadSchemaType.Integer +); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/go.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/go.md new file mode 100644 index 000000000..5a623ccc3 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/go.md @@ -0,0 +1,7 @@ +```go +client.CreateFieldIndex(context.Background(), &qdrant.CreateFieldIndexCollection{ + CollectionName: collectionName, + FieldName: "year", + FieldType: qdrant.FieldType_FieldTypeInteger.Enum(), +}) +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/java.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/java.md new file mode 100644 index 000000000..0c22e01db --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/java.md @@ -0,0 +1,12 @@ +```java +client + .createPayloadIndexAsync( + COLLECTION_NAME, + "year", + PayloadSchemaType.Integer, + null, + true, + null, + null) + .get(); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-payload-index/generated/python.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/python.md similarity index 100% rename from qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/create-payload-index/generated/python.md rename to qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/python.md diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/rust.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/rust.md new file mode 100644 index 000000000..5b4f021e0 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/rust.md @@ -0,0 +1,8 @@ +```rust +client + .create_field_index( + CreateFieldIndexCollectionBuilder::new(collection_name, "year", FieldType::Integer) + .wait(true), + ) + .await?; +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/typescript.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/typescript.md new file mode 100644 index 000000000..b44eb8c5a --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/create-payload-index/typescript.md @@ -0,0 +1,6 @@ +```typescript +await client.createPayloadIndex(collectionName, { + field_name: "year", + field_schema: "integer", +}); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/csharp.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/csharp.md new file mode 100644 index 000000000..7ef6ca4e8 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/csharp.md @@ -0,0 +1,104 @@ +```csharp +using Qdrant.Client; +using Qdrant.Client.Grpc; +using static Qdrant.Client.Grpc.Conditions; + + var client = new QdrantClient( + host: QDRANT_URL, + port: 6334, + https: true, + apiKey: QDRANT_API_KEY + ); + + string COLLECTION_NAME = "my_books"; + + await client.CreateCollectionAsync( + collectionName: COLLECTION_NAME, + vectorsConfig: new VectorParams { Size = 384, Distance = Distance.Cosine } + ); + + var payloads = new List> + { + new() { ["name"] = "The Time Machine", ["description"] = "A man travels through time and witnesses the evolution of humanity.", ["author"] = "H.G. Wells", ["year"] = 1895 }, + new() { ["name"] = "Ender's Game", ["description"] = "A young boy is trained to become a military leader in a war against an alien race.", ["author"] = "Orson Scott Card", ["year"] = 1985 }, + new() { ["name"] = "Brave New World", ["description"] = "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", ["author"] = "Aldous Huxley", ["year"] = 1932 }, + new() { ["name"] = "The Hitchhiker's Guide to the Galaxy", ["description"] = "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", ["author"] = "Douglas Adams", ["year"] = 1979 }, + new() { ["name"] = "Dune", ["description"] = "A desert planet is the site of political intrigue and power struggles.", ["author"] = "Frank Herbert", ["year"] = 1965 }, + new() { ["name"] = "Foundation", ["description"] = "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", ["author"] = "Isaac Asimov", ["year"] = 1951 }, + new() { ["name"] = "Snow Crash", ["description"] = "A futuristic world where the internet has evolved into a virtual reality metaverse.", ["author"] = "Neal Stephenson", ["year"] = 1992 }, + new() { ["name"] = "Neuromancer", ["description"] = "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", ["author"] = "William Gibson", ["year"] = 1984 }, + new() { ["name"] = "The War of the Worlds", ["description"] = "A Martian invasion of Earth throws humanity into chaos.", ["author"] = "H.G. Wells", ["year"] = 1898 }, + new() { ["name"] = "The Hunger Games", ["description"] = "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", ["author"] = "Suzanne Collins", ["year"] = 2008 }, + new() { ["name"] = "The Andromeda Strain", ["description"] = "A deadly virus from outer space threatens to wipe out humanity.", ["author"] = "Michael Crichton", ["year"] = 1969 }, + new() { ["name"] = "The Left Hand of Darkness", ["description"] = "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", ["author"] = "Ursula K. Le Guin", ["year"] = 1969 }, + new() { ["name"] = "The Three-Body Problem", ["description"] = "Humans encounter an alien civilization that lives in a dying system.", ["author"] = "Liu Cixin", ["year"] = 2008 } + }; + + string EMBEDDING_MODEL = "sentence-transformers/all-minilm-l6-v2"; + + var points = new List(); + + for (ulong idx = 0; idx < (ulong)payloads.Count; idx++) + { + var payload = payloads[(int)idx]; + string description = payload["description"].StringValue; + + var point = new PointStruct + { + Id = idx, + Vectors = new Document + { + Text = description, + Model = EMBEDDING_MODEL + }, + Payload = { payload } + }; + + points.Add(point); + } + + await client.UpsertAsync( + collectionName: COLLECTION_NAME, + points: points + ); + + var hits = await client.QueryAsync( + collectionName: COLLECTION_NAME, + query: new Document + { + Text = "alien invasion", + Model = EMBEDDING_MODEL + }, + limit: 3 + ); + + foreach (var hit in hits) + { + Console.WriteLine($"{hit.Payload} score: {hit.Score}"); + } + + await client.CreatePayloadIndexAsync( + collectionName: COLLECTION_NAME, + fieldName: "year", + schemaType: PayloadSchemaType.Integer + ); + + var filteredHits = await client.QueryAsync( + collectionName: COLLECTION_NAME, + query: new Document + { + Text = "alien invasion", + Model = EMBEDDING_MODEL + }, + filter: new Filter + { + Must = { Range("year", new Qdrant.Client.Grpc.Range { Gte = 2000.0 }) } + }, + limit: 1 + ); + + foreach (var hit in filteredHits) + { + Console.WriteLine($"{hit.Payload} score: {hit.Score}"); + } +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/go.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/go.md new file mode 100644 index 000000000..5507ab627 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/go.md @@ -0,0 +1,163 @@ +```go +import ( + "context" + "fmt" + + "github.com/qdrant/go-client/qdrant" +) + +client, err := qdrant.NewClient(&qdrant.Config{ + Host: QDRANT_URL, + APIKey: QDRANT_API_KEY, + UseTLS: true, +}) + +collectionName := "my_books" + +client.CreateCollection(context.Background(), &qdrant.CreateCollection{ + CollectionName: collectionName, + VectorsConfig: qdrant.NewVectorsConfig(&qdrant.VectorParams{ + Size: 384, // Vector size is defined by used model + Distance: qdrant.Distance_Cosine, + }), +}) + +documents := []map[string]any{ + { + "name": "The Time Machine", + "description": "A man travels through time and witnesses the evolution of humanity.", + "author": "H.G. Wells", + "year": 1895, + }, + { + "name": "Ender's Game", + "description": "A young boy is trained to become a military leader in a war against an alien race.", + "author": "Orson Scott Card", + "year": 1985, + }, + { + "name": "Brave New World", + "description": "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", + "author": "Aldous Huxley", + "year": 1932, + }, + { + "name": "The Hitchhiker's Guide to the Galaxy", + "description": "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", + "author": "Douglas Adams", + "year": 1979, + }, + { + "name": "Dune", + "description": "A desert planet is the site of political intrigue and power struggles.", + "author": "Frank Herbert", + "year": 1965, + }, + { + "name": "Foundation", + "description": "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", + "author": "Isaac Asimov", + "year": 1951, + }, + { + "name": "Snow Crash", + "description": "A futuristic world where the internet has evolved into a virtual reality metaverse.", + "author": "Neal Stephenson", + "year": 1992, + }, + { + "name": "Neuromancer", + "description": "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", + "author": "William Gibson", + "year": 1984, + }, + { + "name": "The War of the Worlds", + "description": "A Martian invasion of Earth throws humanity into chaos.", + "author": "H.G. Wells", + "year": 1898, + }, + { + "name": "The Hunger Games", + "description": "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", + "author": "Suzanne Collins", + "year": 2008, + }, + { + "name": "The Andromeda Strain", + "description": "A deadly virus from outer space threatens to wipe out humanity.", + "author": "Michael Crichton", + "year": 1969, + }, + { + "name": "The Left Hand of Darkness", + "description": "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", + "author": "Ursula K. Le Guin", + "year": 1969, + }, + { + "name": "The Three-Body Problem", + "description": "Humans encounter an alien civilization that lives in a dying system.", + "author": "Liu Cixin", + "year": 2008, + }, +} + +embeddingModel := "sentence-transformers/all-minilm-l6-v2" + +points := make([]*qdrant.PointStruct, len(documents)) +for idx, doc := range documents { + points[idx] = &qdrant.PointStruct{ + Id: qdrant.NewIDNum(uint64(idx)), + Vectors: qdrant.NewVectorsDocument(&qdrant.Document{ + Text: doc["description"].(string), + Model: embeddingModel, + }), + Payload: qdrant.NewValueMap(doc), + } +} + +client.Upsert(context.Background(), &qdrant.UpsertPoints{ + CollectionName: collectionName, + Points: points, +}) + +queryResult, err := client.Query(context.Background(), &qdrant.QueryPoints{ + CollectionName: collectionName, + Query: qdrant.NewQueryDocument(&qdrant.Document{ + Text: "alien invasion", + Model: embeddingModel, + }), + Limit: qdrant.PtrOf(uint64(3)), +}) + +for _, hit := range queryResult { + fmt.Println(hit.Payload, "score:", hit.Score) +} + +client.CreateFieldIndex(context.Background(), &qdrant.CreateFieldIndexCollection{ + CollectionName: collectionName, + FieldName: "year", + FieldType: qdrant.FieldType_FieldTypeInteger.Enum(), +}) + +queryResultFiltered, err := client.Query(context.Background(), &qdrant.QueryPoints{ + CollectionName: collectionName, + Query: qdrant.NewQueryDocument(&qdrant.Document{ + Text: "alien invasion", + Model: embeddingModel, + }), + Filter: &qdrant.Filter{ + Must: []*qdrant.Condition{ + qdrant.NewRange("year", &qdrant.Range{ + Gte: qdrant.PtrOf(2000.0), + }), + }, + }, + Limit: qdrant.PtrOf(uint64(1)), +}) + +for _, hit := range queryResultFiltered { + fmt.Println(hit.Payload, "score:", hit.Score) +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/java.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/java.md new file mode 100644 index 000000000..5ced1c9b1 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/java.md @@ -0,0 +1,191 @@ +```java +import static io.qdrant.client.ConditionFactory.range; +import static io.qdrant.client.PointIdFactory.id; +import static io.qdrant.client.QueryFactory.nearest; +import static io.qdrant.client.ValueFactory.value; +import static io.qdrant.client.VectorFactory.vector; +import static io.qdrant.client.VectorsFactory.vectors; + +import io.qdrant.client.QdrantClient; +import io.qdrant.client.QdrantGrpcClient; +import io.qdrant.client.grpc.Collections.Distance; +import io.qdrant.client.grpc.Collections.PayloadSchemaType; +import io.qdrant.client.grpc.Collections.VectorParams; +import io.qdrant.client.grpc.Common.Filter; +import io.qdrant.client.grpc.Common.Range; +import io.qdrant.client.grpc.JsonWithInt.Value; +import io.qdrant.client.grpc.Points.Document; +import io.qdrant.client.grpc.Points.PointStruct; +import io.qdrant.client.grpc.Points.QueryPoints; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +QdrantClient client = + new QdrantClient( + QdrantGrpcClient.newBuilder(QDRANT_URL, 6334, true) + .withApiKey(QDRANT_API_KEY) + .build()); + +String COLLECTION_NAME = "my_books"; + +client.createCollectionAsync(COLLECTION_NAME, + VectorParams.newBuilder().setDistance(Distance.Cosine).setSize(384).build()).get(); + +List> payloads = List.of( + Map.of( + "name", value("The Time Machine"), + "description", value("A man travels through time and witnesses the evolution of humanity."), + "author", value("H.G. Wells"), + "year", value(1895)), + Map.of( + "name", value("Ender's Game"), + "description", + value("A young boy is trained to become a military leader in a war against an alien race."), + "author", value("Orson Scott Card"), + "year", value(1985)), + Map.of( + "name", value("Brave New World"), + "description", + value( + "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy."), + "author", value("Aldous Huxley"), + "year", value(1932)), + Map.of( + "name", value("The Hitchhiker's Guide to the Galaxy"), + "description", + value( + "A comedic science fiction series following the misadventures of an unwitting human and his alien friend."), + "author", value("Douglas Adams"), + "year", value(1979)), + Map.of( + "name", value("Dune"), + "description", value("A desert planet is the site of political intrigue and power struggles."), + "author", value("Frank Herbert"), + "year", value(1965)), + Map.of( + "name", value("Foundation"), + "description", + value( + "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse."), + "author", value("Isaac Asimov"), + "year", value(1951)), + Map.of( + "name", value("Snow Crash"), + "description", + value("A futuristic world where the internet has evolved into a virtual reality metaverse."), + "author", value("Neal Stephenson"), + "year", value(1992)), + Map.of( + "name", value("Neuromancer"), + "description", + value( + "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue."), + "author", value("William Gibson"), + "year", value(1984)), + Map.of( + "name", value("The War of the Worlds"), + "description", value("A Martian invasion of Earth throws humanity into chaos."), + "author", value("H.G. Wells"), + "year", value(1898)), + Map.of( + "name", value("The Hunger Games"), + "description", + value("A dystopian society where teenagers are forced to fight to the death in a televised spectacle."), + "author", value("Suzanne Collins"), + "year", value(2008)), + Map.of( + "name", value("The Andromeda Strain"), + "description", value("A deadly virus from outer space threatens to wipe out humanity."), + "author", value("Michael Crichton"), + "year", value(1969)), + Map.of( + "name", value("The Left Hand of Darkness"), + "description", + value( + "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will."), + "author", value("Ursula K. Le Guin"), + "year", value(1969)), + Map.of( + "name", value("The Three-Body Problem"), + "description", value("Humans encounter an alien civilization that lives in a dying system."), + "author", value("Liu Cixin"), + "year", value(2008))); + +String EMBEDDING_MODEL = "sentence-transformers/all-minilm-l6-v2"; + +List points = new ArrayList<>(); + +for (int idx = 0; idx < payloads.size(); idx++) { + Map payload = payloads.get(idx); + String description = payload.get("description").getStringValue(); + + PointStruct point = + PointStruct.newBuilder() + .setId(id((long) idx)) + .setVectors( + vectors( + vector( + Document.newBuilder() + .setText(description) + .setModel(EMBEDDING_MODEL) + .build()))) + .putAllPayload(payload) + .build(); + + points.add(point); +} + +client.upsertAsync(COLLECTION_NAME, points).get(); + +QueryPoints request = + QueryPoints.newBuilder() + .setCollectionName(COLLECTION_NAME) + .setQuery( + nearest( + Document.newBuilder() + .setText("alien invasion") + .setModel(EMBEDDING_MODEL) + .build())) + .setLimit(3) + .build(); + +var hits = client.queryAsync(request).get(); + +for (var hit : hits) { + System.out.println(hit.getPayloadMap() + " score: " + hit.getScore()); +} + +client + .createPayloadIndexAsync( + COLLECTION_NAME, + "year", + PayloadSchemaType.Integer, + null, + true, + null, + null) + .get(); + +QueryPoints filteredRequest = + QueryPoints.newBuilder() + .setCollectionName(COLLECTION_NAME) + .setQuery( + nearest( + Document.newBuilder() + .setText("alien invasion") + .setModel(EMBEDDING_MODEL) + .build())) + .setFilter( + Filter.newBuilder() + .addMust(range("year", Range.newBuilder().setGte(2000.0).build())) + .build()) + .setLimit(1) + .build(); + +var filteredHits = client.queryAsync(filteredRequest).get(); + +for (var hit : filteredHits) { + System.out.println(hit.getPayloadMap() + " score: " + hit.getScore()); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-data/python.py b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/python.md similarity index 64% rename from qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-data/python.py rename to qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/python.md index e36c487d1..8d5b8047d 100644 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-data/python.py +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/python.md @@ -1,3 +1,22 @@ +```python +from qdrant_client import QdrantClient, models + +client = QdrantClient( + url=QDRANT_URL, + api_key=QDRANT_API_KEY, + cloud_inference=True +) + +COLLECTION_NAME="my_books" + +client.create_collection( + collection_name=COLLECTION_NAME, + vectors_config=models.VectorParams( + size=384, # Vector size is defined by used model + distance=models.Distance.COSINE, + ), +) + documents = [ { "name": "The Time Machine", @@ -77,4 +96,55 @@ documents = [ "author": "Liu Cixin", "year": 2008, }, -] \ No newline at end of file +] + +EMBEDDING_MODEL="sentence-transformers/all-minilm-l6-v2" + +client.upload_points( + collection_name=COLLECTION_NAME, + points=[ + models.PointStruct( + id=idx, + vector=models.Document( + text=doc["description"], + model=EMBEDDING_MODEL + ), + payload=doc + ) + for idx, doc in enumerate(documents) + ], +) + +hits = client.query_points( + collection_name=COLLECTION_NAME, + query=models.Document( + text="alien invasion", + model=EMBEDDING_MODEL + ), + limit=3, +).points + +for hit in hits: + print(hit.payload, "score:", hit.score) + +client.create_payload_index( + collection_name=COLLECTION_NAME, + field_name="year", + field_schema=models.PayloadSchemaType.INTEGER, +) + +hits = client.query_points( + collection_name=COLLECTION_NAME, + query=models.Document( + text="alien invasion", + model=EMBEDDING_MODEL + ), + query_filter=models.Filter( + must=[models.FieldCondition(key="year", range=models.Range(gte=2000))] + ), + limit=1, +).points + +for hit in hits: + print(hit.payload, "score:", hit.score) +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/csharp.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/csharp.md new file mode 100644 index 000000000..1c53f7750 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/csharp.md @@ -0,0 +1,16 @@ +```csharp +var hits = await client.QueryAsync( + collectionName: COLLECTION_NAME, + query: new Document + { + Text = "alien invasion", + Model = EMBEDDING_MODEL + }, + limit: 3 +); + +foreach (var hit in hits) +{ + Console.WriteLine($"{hit.Payload} score: {hit.Score}"); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/go.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/go.md new file mode 100644 index 000000000..3ef707789 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/go.md @@ -0,0 +1,14 @@ +```go +queryResult, err := client.Query(context.Background(), &qdrant.QueryPoints{ + CollectionName: collectionName, + Query: qdrant.NewQueryDocument(&qdrant.Document{ + Text: "alien invasion", + Model: embeddingModel, + }), + Limit: qdrant.PtrOf(uint64(3)), +}) + +for _, hit := range queryResult { + fmt.Println(hit.Payload, "score:", hit.Score) +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/java.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/java.md new file mode 100644 index 000000000..7f637db2c --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/java.md @@ -0,0 +1,19 @@ +```java +QueryPoints request = + QueryPoints.newBuilder() + .setCollectionName(COLLECTION_NAME) + .setQuery( + nearest( + Document.newBuilder() + .setText("alien invasion") + .setModel(EMBEDDING_MODEL) + .build())) + .setLimit(3) + .build(); + +var hits = client.queryAsync(request).get(); + +for (var hit : hits) { + System.out.println(hit.getPayloadMap() + " score: " + hit.getScore()); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-engine/generated/python.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/python.md similarity index 100% rename from qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-engine/generated/python.md rename to qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/python.md diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/rust.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/rust.md new file mode 100644 index 000000000..e09236b18 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/rust.md @@ -0,0 +1,17 @@ +```rust +let query_result = client + .query( + QueryPointsBuilder::new(collection_name) + .query(Query::new_nearest(Document::new( + "alien invasion", + embedding_model, + ))) + .limit(3) + .with_payload(true), + ) + .await?; + +for hit in query_result.result { + println!("{:?} score: {}", hit.payload, hit.score); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/typescript.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/typescript.md new file mode 100644 index 000000000..4a2abed01 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-engine/typescript.md @@ -0,0 +1,13 @@ +```typescript +const queryResult = await client.query(collectionName, { + query: { + text: "alien invasion", + model: embeddingModel, + }, + limit: 3, +}); + +for (const hit of queryResult.points) { + console.log(hit.payload, "score:", hit.score); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/csharp.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/csharp.md new file mode 100644 index 000000000..58ab645f0 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/csharp.md @@ -0,0 +1,20 @@ +```csharp +var filteredHits = await client.QueryAsync( + collectionName: COLLECTION_NAME, + query: new Document + { + Text = "alien invasion", + Model = EMBEDDING_MODEL + }, + filter: new Filter + { + Must = { Range("year", new Qdrant.Client.Grpc.Range { Gte = 2000.0 }) } + }, + limit: 1 +); + +foreach (var hit in filteredHits) +{ + Console.WriteLine($"{hit.Payload} score: {hit.Score}"); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/go.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/go.md new file mode 100644 index 000000000..a35ddc082 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/go.md @@ -0,0 +1,21 @@ +```go +queryResultFiltered, err := client.Query(context.Background(), &qdrant.QueryPoints{ + CollectionName: collectionName, + Query: qdrant.NewQueryDocument(&qdrant.Document{ + Text: "alien invasion", + Model: embeddingModel, + }), + Filter: &qdrant.Filter{ + Must: []*qdrant.Condition{ + qdrant.NewRange("year", &qdrant.Range{ + Gte: qdrant.PtrOf(2000.0), + }), + }, + }, + Limit: qdrant.PtrOf(uint64(1)), +}) + +for _, hit := range queryResultFiltered { + fmt.Println(hit.Payload, "score:", hit.Score) +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/java.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/java.md new file mode 100644 index 000000000..a0d685def --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/java.md @@ -0,0 +1,23 @@ +```java +QueryPoints filteredRequest = + QueryPoints.newBuilder() + .setCollectionName(COLLECTION_NAME) + .setQuery( + nearest( + Document.newBuilder() + .setText("alien invasion") + .setModel(EMBEDDING_MODEL) + .build())) + .setFilter( + Filter.newBuilder() + .addMust(range("year", Range.newBuilder().setGte(2000.0).build())) + .build()) + .setLimit(1) + .build(); + +var filteredHits = client.queryAsync(filteredRequest).get(); + +for (var hit : filteredHits) { + System.out.println(hit.getPayloadMap() + " score: " + hit.getScore()); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-with-filter/generated/python.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/python.md similarity index 100% rename from qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-with-filter/generated/python.md rename to qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/python.md diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/rust.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/rust.md new file mode 100644 index 000000000..ad728443b --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/rust.md @@ -0,0 +1,24 @@ +```rust +let query_result_filtered = client + .query( + QueryPointsBuilder::new(collection_name) + .query(Query::new_nearest(Document::new( + "alien invasion", + embedding_model, + ))) + .filter(Filter::must([Condition::range( + "year", + Range { + gte: Some(2000.0), + ..Default::default() + }, + )])) + .limit(1) + .with_payload(true), + ) + .await?; + +for hit in query_result_filtered.result { + println!("{:?} score: {}", hit.payload, hit.score); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/typescript.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/typescript.md new file mode 100644 index 000000000..b55b3a9cb --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/query-with-filter/typescript.md @@ -0,0 +1,23 @@ +```typescript +const queryResultFiltered = await client.query(collectionName, { + query: { + text: "alien invasion", + model: embeddingModel, + }, + filter: { + must: [ + { + key: "year", + range: { + gte: 2000, + }, + }, + ], + }, + limit: 1, +}); + +for (const hit of queryResultFiltered.points) { + console.log(hit.payload, "score:", hit.score); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/rust.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/rust.md new file mode 100644 index 000000000..2743cb8a5 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/rust.md @@ -0,0 +1,105 @@ +```rust +use qdrant_client::qdrant::{ + Condition, CreateCollectionBuilder, CreateFieldIndexCollectionBuilder, Distance, Document, + FieldType, Filter, PointStruct, Query, QueryPointsBuilder, Range, UpsertPointsBuilder, VectorParamsBuilder, +}; +use qdrant_client::Qdrant; + +let client = Qdrant::from_url(QDRANT_URL) + .api_key(QDRANT_API_KEY) + .build()?; + +let collection_name = "my_books"; + +client + .create_collection( + CreateCollectionBuilder::new(collection_name) + .vectors_config(VectorParamsBuilder::new(384, Distance::Cosine)), // Vector size is defined by used model + ) + .await?; + +let documents = [ + ("The Time Machine", "A man travels through time and witnesses the evolution of humanity.", "H.G. Wells", 1895), + ("Ender's Game", "A young boy is trained to become a military leader in a war against an alien race.", "Orson Scott Card", 1985), + ("Brave New World", "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", "Aldous Huxley", 1932), + ("The Hitchhiker's Guide to the Galaxy", "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", "Douglas Adams", 1979), + ("Dune", "A desert planet is the site of political intrigue and power struggles.", "Frank Herbert", 1965), + ("Foundation", "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", "Isaac Asimov", 1951), + ("Snow Crash", "A futuristic world where the internet has evolved into a virtual reality metaverse.", "Neal Stephenson", 1992), + ("Neuromancer", "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", "William Gibson", 1984), + ("The War of the Worlds", "A Martian invasion of Earth throws humanity into chaos.", "H.G. Wells", 1898), + ("The Hunger Games", "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", "Suzanne Collins", 2008), + ("The Andromeda Strain", "A deadly virus from outer space threatens to wipe out humanity.", "Michael Crichton", 1969), + ("The Left Hand of Darkness", "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", "Ursula K. Le Guin", 1969), + ("The Three-Body Problem", "Humans encounter an alien civilization that lives in a dying system.", "Liu Cixin", 2008), +]; + +let embedding_model = "sentence-transformers/all-minilm-l6-v2"; + +let points: Vec = documents + .iter() + .enumerate() + .map(|(idx, (name, description, author, year))| { + PointStruct::new( + idx as u64, + Document::new(*description, embedding_model), + [ + ("name", (*name).into()), + ("description", (*description).into()), + ("author", (*author).into()), + ("year", (*year).into()), + ], + ) + }) + .collect(); + +client + .upsert_points(UpsertPointsBuilder::new(collection_name, points)) + .await?; + +let query_result = client + .query( + QueryPointsBuilder::new(collection_name) + .query(Query::new_nearest(Document::new( + "alien invasion", + embedding_model, + ))) + .limit(3) + .with_payload(true), + ) + .await?; + +for hit in query_result.result { + println!("{:?} score: {}", hit.payload, hit.score); +} + +client + .create_field_index( + CreateFieldIndexCollectionBuilder::new(collection_name, "year", FieldType::Integer) + .wait(true), + ) + .await?; + +let query_result_filtered = client + .query( + QueryPointsBuilder::new(collection_name) + .query(Query::new_nearest(Document::new( + "alien invasion", + embedding_model, + ))) + .filter(Filter::must([Condition::range( + "year", + Range { + gte: Some(2000.0), + ..Default::default() + }, + )])) + .limit(1) + .with_payload(true), + ) + .await?; + +for hit in query_result_filtered.result { + println!("{:?} score: {}", hit.payload, hit.score); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/typescript.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/typescript.md new file mode 100644 index 000000000..22a656d05 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/typescript.md @@ -0,0 +1,85 @@ +```typescript +import { QdrantClient } from "@qdrant/js-client-rest"; + +const client = new QdrantClient({ + url: QDRANT_URL, + apiKey: QDRANT_API_KEY, +}); + +const collectionName = "my_books"; + +await client.createCollection(collectionName, { + vectors: { + size: 384, // Vector size is defined by used model + distance: "Cosine", + }, +}); + +const documents = [ + { name: "The Time Machine", description: "A man travels through time and witnesses the evolution of humanity.", author: "H.G. Wells", year: 1895 }, + { name: "Ender's Game", description: "A young boy is trained to become a military leader in a war against an alien race.", author: "Orson Scott Card", year: 1985 }, + { name: "Brave New World", description: "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", author: "Aldous Huxley", year: 1932 }, + { name: "The Hitchhiker's Guide to the Galaxy", description: "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", author: "Douglas Adams", year: 1979 }, + { name: "Dune", description: "A desert planet is the site of political intrigue and power struggles.", author: "Frank Herbert", year: 1965 }, + { name: "Foundation", description: "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", author: "Isaac Asimov", year: 1951 }, + { name: "Snow Crash", description: "A futuristic world where the internet has evolved into a virtual reality metaverse.", author: "Neal Stephenson", year: 1992 }, + { name: "Neuromancer", description: "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", author: "William Gibson", year: 1984 }, + { name: "The War of the Worlds", description: "A Martian invasion of Earth throws humanity into chaos.", author: "H.G. Wells", year: 1898 }, + { name: "The Hunger Games", description: "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", author: "Suzanne Collins", year: 2008 }, + { name: "The Andromeda Strain", description: "A deadly virus from outer space threatens to wipe out humanity.", author: "Michael Crichton", year: 1969 }, + { name: "The Left Hand of Darkness", description: "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", author: "Ursula K. Le Guin", year: 1969 }, + { name: "The Three-Body Problem", description: "Humans encounter an alien civilization that lives in a dying system.", author: "Liu Cixin", year: 2008 }, +]; + +const embeddingModel = "sentence-transformers/all-minilm-l6-v2"; + +const points = documents.map((doc, idx) => ({ + id: idx, + vector: { + text: doc.description, + model: embeddingModel, + }, + payload: doc, +})); + +await client.upsert(collectionName, { points }); + +const queryResult = await client.query(collectionName, { + query: { + text: "alien invasion", + model: embeddingModel, + }, + limit: 3, +}); + +for (const hit of queryResult.points) { + console.log(hit.payload, "score:", hit.score); +} + +await client.createPayloadIndex(collectionName, { + field_name: "year", + field_schema: "integer", +}); + +const queryResultFiltered = await client.query(collectionName, { + query: { + text: "alien invasion", + model: embeddingModel, + }, + filter: { + must: [ + { + key: "year", + range: { + gte: 2000, + }, + }, + ], + }, + limit: 1, +}); + +for (const hit of queryResultFiltered.points) { + console.log(hit.payload, "score:", hit.score); +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/csharp.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/csharp.md new file mode 100644 index 000000000..a3bf353d0 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/csharp.md @@ -0,0 +1,18 @@ +```csharp +var payloads = new List> +{ + new() { ["name"] = "The Time Machine", ["description"] = "A man travels through time and witnesses the evolution of humanity.", ["author"] = "H.G. Wells", ["year"] = 1895 }, + new() { ["name"] = "Ender's Game", ["description"] = "A young boy is trained to become a military leader in a war against an alien race.", ["author"] = "Orson Scott Card", ["year"] = 1985 }, + new() { ["name"] = "Brave New World", ["description"] = "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", ["author"] = "Aldous Huxley", ["year"] = 1932 }, + new() { ["name"] = "The Hitchhiker's Guide to the Galaxy", ["description"] = "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", ["author"] = "Douglas Adams", ["year"] = 1979 }, + new() { ["name"] = "Dune", ["description"] = "A desert planet is the site of political intrigue and power struggles.", ["author"] = "Frank Herbert", ["year"] = 1965 }, + new() { ["name"] = "Foundation", ["description"] = "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", ["author"] = "Isaac Asimov", ["year"] = 1951 }, + new() { ["name"] = "Snow Crash", ["description"] = "A futuristic world where the internet has evolved into a virtual reality metaverse.", ["author"] = "Neal Stephenson", ["year"] = 1992 }, + new() { ["name"] = "Neuromancer", ["description"] = "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", ["author"] = "William Gibson", ["year"] = 1984 }, + new() { ["name"] = "The War of the Worlds", ["description"] = "A Martian invasion of Earth throws humanity into chaos.", ["author"] = "H.G. Wells", ["year"] = 1898 }, + new() { ["name"] = "The Hunger Games", ["description"] = "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", ["author"] = "Suzanne Collins", ["year"] = 2008 }, + new() { ["name"] = "The Andromeda Strain", ["description"] = "A deadly virus from outer space threatens to wipe out humanity.", ["author"] = "Michael Crichton", ["year"] = 1969 }, + new() { ["name"] = "The Left Hand of Darkness", ["description"] = "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", ["author"] = "Ursula K. Le Guin", ["year"] = 1969 }, + new() { ["name"] = "The Three-Body Problem", ["description"] = "Humans encounter an alien civilization that lives in a dying system.", ["author"] = "Liu Cixin", ["year"] = 2008 } +}; +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/go.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/go.md new file mode 100644 index 000000000..7c726ba2c --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/go.md @@ -0,0 +1,82 @@ +```go +documents := []map[string]any{ + { + "name": "The Time Machine", + "description": "A man travels through time and witnesses the evolution of humanity.", + "author": "H.G. Wells", + "year": 1895, + }, + { + "name": "Ender's Game", + "description": "A young boy is trained to become a military leader in a war against an alien race.", + "author": "Orson Scott Card", + "year": 1985, + }, + { + "name": "Brave New World", + "description": "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", + "author": "Aldous Huxley", + "year": 1932, + }, + { + "name": "The Hitchhiker's Guide to the Galaxy", + "description": "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", + "author": "Douglas Adams", + "year": 1979, + }, + { + "name": "Dune", + "description": "A desert planet is the site of political intrigue and power struggles.", + "author": "Frank Herbert", + "year": 1965, + }, + { + "name": "Foundation", + "description": "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", + "author": "Isaac Asimov", + "year": 1951, + }, + { + "name": "Snow Crash", + "description": "A futuristic world where the internet has evolved into a virtual reality metaverse.", + "author": "Neal Stephenson", + "year": 1992, + }, + { + "name": "Neuromancer", + "description": "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", + "author": "William Gibson", + "year": 1984, + }, + { + "name": "The War of the Worlds", + "description": "A Martian invasion of Earth throws humanity into chaos.", + "author": "H.G. Wells", + "year": 1898, + }, + { + "name": "The Hunger Games", + "description": "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", + "author": "Suzanne Collins", + "year": 2008, + }, + { + "name": "The Andromeda Strain", + "description": "A deadly virus from outer space threatens to wipe out humanity.", + "author": "Michael Crichton", + "year": 1969, + }, + { + "name": "The Left Hand of Darkness", + "description": "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", + "author": "Ursula K. Le Guin", + "year": 1969, + }, + { + "name": "The Three-Body Problem", + "description": "Humans encounter an alien civilization that lives in a dying system.", + "author": "Liu Cixin", + "year": 2008, + }, +} +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/java.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/java.md new file mode 100644 index 000000000..255a27f76 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/java.md @@ -0,0 +1,81 @@ +```java +List> payloads = List.of( + Map.of( + "name", value("The Time Machine"), + "description", value("A man travels through time and witnesses the evolution of humanity."), + "author", value("H.G. Wells"), + "year", value(1895)), + Map.of( + "name", value("Ender's Game"), + "description", + value("A young boy is trained to become a military leader in a war against an alien race."), + "author", value("Orson Scott Card"), + "year", value(1985)), + Map.of( + "name", value("Brave New World"), + "description", + value( + "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy."), + "author", value("Aldous Huxley"), + "year", value(1932)), + Map.of( + "name", value("The Hitchhiker's Guide to the Galaxy"), + "description", + value( + "A comedic science fiction series following the misadventures of an unwitting human and his alien friend."), + "author", value("Douglas Adams"), + "year", value(1979)), + Map.of( + "name", value("Dune"), + "description", value("A desert planet is the site of political intrigue and power struggles."), + "author", value("Frank Herbert"), + "year", value(1965)), + Map.of( + "name", value("Foundation"), + "description", + value( + "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse."), + "author", value("Isaac Asimov"), + "year", value(1951)), + Map.of( + "name", value("Snow Crash"), + "description", + value("A futuristic world where the internet has evolved into a virtual reality metaverse."), + "author", value("Neal Stephenson"), + "year", value(1992)), + Map.of( + "name", value("Neuromancer"), + "description", + value( + "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue."), + "author", value("William Gibson"), + "year", value(1984)), + Map.of( + "name", value("The War of the Worlds"), + "description", value("A Martian invasion of Earth throws humanity into chaos."), + "author", value("H.G. Wells"), + "year", value(1898)), + Map.of( + "name", value("The Hunger Games"), + "description", + value("A dystopian society where teenagers are forced to fight to the death in a televised spectacle."), + "author", value("Suzanne Collins"), + "year", value(2008)), + Map.of( + "name", value("The Andromeda Strain"), + "description", value("A deadly virus from outer space threatens to wipe out humanity."), + "author", value("Michael Crichton"), + "year", value(1969)), + Map.of( + "name", value("The Left Hand of Darkness"), + "description", + value( + "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will."), + "author", value("Ursula K. Le Guin"), + "year", value(1969)), + Map.of( + "name", value("The Three-Body Problem"), + "description", value("Humans encounter an alien civilization that lives in a dying system."), + "author", value("Liu Cixin"), + "year", value(2008))); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-data/generated/python.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/python.md similarity index 100% rename from qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-data/generated/python.md rename to qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/python.md diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/rust.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/rust.md new file mode 100644 index 000000000..a0a1019cd --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/rust.md @@ -0,0 +1,17 @@ +```rust +let documents = [ + ("The Time Machine", "A man travels through time and witnesses the evolution of humanity.", "H.G. Wells", 1895), + ("Ender's Game", "A young boy is trained to become a military leader in a war against an alien race.", "Orson Scott Card", 1985), + ("Brave New World", "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", "Aldous Huxley", 1932), + ("The Hitchhiker's Guide to the Galaxy", "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", "Douglas Adams", 1979), + ("Dune", "A desert planet is the site of political intrigue and power struggles.", "Frank Herbert", 1965), + ("Foundation", "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", "Isaac Asimov", 1951), + ("Snow Crash", "A futuristic world where the internet has evolved into a virtual reality metaverse.", "Neal Stephenson", 1992), + ("Neuromancer", "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", "William Gibson", 1984), + ("The War of the Worlds", "A Martian invasion of Earth throws humanity into chaos.", "H.G. Wells", 1898), + ("The Hunger Games", "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", "Suzanne Collins", 2008), + ("The Andromeda Strain", "A deadly virus from outer space threatens to wipe out humanity.", "Michael Crichton", 1969), + ("The Left Hand of Darkness", "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", "Ursula K. Le Guin", 1969), + ("The Three-Body Problem", "Humans encounter an alien civilization that lives in a dying system.", "Liu Cixin", 2008), +]; +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/typescript.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/typescript.md new file mode 100644 index 000000000..e543a56dc --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-data/typescript.md @@ -0,0 +1,17 @@ +```typescript +const documents = [ + { name: "The Time Machine", description: "A man travels through time and witnesses the evolution of humanity.", author: "H.G. Wells", year: 1895 }, + { name: "Ender's Game", description: "A young boy is trained to become a military leader in a war against an alien race.", author: "Orson Scott Card", year: 1985 }, + { name: "Brave New World", description: "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", author: "Aldous Huxley", year: 1932 }, + { name: "The Hitchhiker's Guide to the Galaxy", description: "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", author: "Douglas Adams", year: 1979 }, + { name: "Dune", description: "A desert planet is the site of political intrigue and power struggles.", author: "Frank Herbert", year: 1965 }, + { name: "Foundation", description: "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", author: "Isaac Asimov", year: 1951 }, + { name: "Snow Crash", description: "A futuristic world where the internet has evolved into a virtual reality metaverse.", author: "Neal Stephenson", year: 1992 }, + { name: "Neuromancer", description: "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", author: "William Gibson", year: 1984 }, + { name: "The War of the Worlds", description: "A Martian invasion of Earth throws humanity into chaos.", author: "H.G. Wells", year: 1898 }, + { name: "The Hunger Games", description: "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", author: "Suzanne Collins", year: 2008 }, + { name: "The Andromeda Strain", description: "A deadly virus from outer space threatens to wipe out humanity.", author: "Michael Crichton", year: 1969 }, + { name: "The Left Hand of Darkness", description: "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", author: "Ursula K. Le Guin", year: 1969 }, + { name: "The Three-Body Problem", description: "Humans encounter an alien civilization that lives in a dying system.", author: "Liu Cixin", year: 2008 }, +]; +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/csharp.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/csharp.md new file mode 100644 index 000000000..0c2bcb253 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/csharp.md @@ -0,0 +1,29 @@ +```csharp +string EMBEDDING_MODEL = "sentence-transformers/all-minilm-l6-v2"; + +var points = new List(); + +for (ulong idx = 0; idx < (ulong)payloads.Count; idx++) +{ + var payload = payloads[(int)idx]; + string description = payload["description"].StringValue; + + var point = new PointStruct + { + Id = idx, + Vectors = new Document + { + Text = description, + Model = EMBEDDING_MODEL + }, + Payload = { payload } + }; + + points.Add(point); +} + +await client.UpsertAsync( + collectionName: COLLECTION_NAME, + points: points +); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/go.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/go.md new file mode 100644 index 000000000..5e14790b4 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/go.md @@ -0,0 +1,20 @@ +```go +embeddingModel := "sentence-transformers/all-minilm-l6-v2" + +points := make([]*qdrant.PointStruct, len(documents)) +for idx, doc := range documents { + points[idx] = &qdrant.PointStruct{ + Id: qdrant.NewIDNum(uint64(idx)), + Vectors: qdrant.NewVectorsDocument(&qdrant.Document{ + Text: doc["description"].(string), + Model: embeddingModel, + }), + Payload: qdrant.NewValueMap(doc), + } +} + +client.Upsert(context.Background(), &qdrant.UpsertPoints{ + CollectionName: collectionName, + Points: points, +}) +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/java.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/java.md new file mode 100644 index 000000000..3263c134a --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/java.md @@ -0,0 +1,27 @@ +```java +String EMBEDDING_MODEL = "sentence-transformers/all-minilm-l6-v2"; + +List points = new ArrayList<>(); + +for (int idx = 0; idx < payloads.size(); idx++) { + Map payload = payloads.get(idx); + String description = payload.get("description").getStringValue(); + + PointStruct point = + PointStruct.newBuilder() + .setId(id((long) idx)) + .setVectors( + vectors( + vector( + Document.newBuilder() + .setText(description) + .setModel(EMBEDDING_MODEL) + .build()))) + .putAllPayload(payload) + .build(); + + points.add(point); +} + +client.upsertAsync(COLLECTION_NAME, points).get(); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/generated/python.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/python.md similarity index 71% rename from qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/generated/python.md rename to qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/python.md index ee030f165..f37d79520 100644 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/generated/python.md +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/python.md @@ -1,5 +1,4 @@ ```python -# Define the embedding model used by Cloud Inference EMBEDDING_MODEL="sentence-transformers/all-minilm-l6-v2" client.upload_points( @@ -9,7 +8,7 @@ client.upload_points( id=idx, vector=models.Document( text=doc["description"], - model=EMBEDDING_MODEL # Cloud Inference generates embeddings with this model + model=EMBEDDING_MODEL ), payload=doc ) diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/rust.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/rust.md new file mode 100644 index 000000000..41bfe0396 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/rust.md @@ -0,0 +1,24 @@ +```rust +let embedding_model = "sentence-transformers/all-minilm-l6-v2"; + +let points: Vec = documents + .iter() + .enumerate() + .map(|(idx, (name, description, author, year))| { + PointStruct::new( + idx as u64, + Document::new(*description, embedding_model), + [ + ("name", (*name).into()), + ("description", (*description).into()), + ("author", (*author).into()), + ("year", (*year).into()), + ], + ) + }) + .collect(); + +client + .upsert_points(UpsertPointsBuilder::new(collection_name, points)) + .await?; +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/typescript.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/typescript.md new file mode 100644 index 000000000..8f153485e --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/generated/upload-points/typescript.md @@ -0,0 +1,14 @@ +```typescript +const embeddingModel = "sentence-transformers/all-minilm-l6-v2"; + +const points = documents.map((doc, idx) => ({ + id: idx, + vector: { + text: doc.description, + model: embeddingModel, + }, + payload: doc, +})); + +await client.upsert(collectionName, { points }); +``` diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/go.go b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/go.go new file mode 100644 index 000000000..5a9989380 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/go.go @@ -0,0 +1,201 @@ +package snippet + +import ( + "context" + "fmt" + + "github.com/qdrant/go-client/qdrant" +) + +func Main() { + // @hide-start + QDRANT_URL := "" + QDRANT_API_KEY := "" + // @hide-end + // @block-start client-connection + client, err := qdrant.NewClient(&qdrant.Config{ + Host: QDRANT_URL, + APIKey: QDRANT_API_KEY, + UseTLS: true, + }) + // @block-end client-connection + + // @hide-start + if err != nil { + panic(err) + } + // @hide-end + + // @block-start create-collection + collectionName := "my_books" + + client.CreateCollection(context.Background(), &qdrant.CreateCollection{ + CollectionName: collectionName, + VectorsConfig: qdrant.NewVectorsConfig(&qdrant.VectorParams{ + Size: 384, // Vector size is defined by used model + Distance: qdrant.Distance_Cosine, + }), + }) + // @block-end create-collection + + // @block-start upload-data + documents := []map[string]any{ + { + "name": "The Time Machine", + "description": "A man travels through time and witnesses the evolution of humanity.", + "author": "H.G. Wells", + "year": 1895, + }, + { + "name": "Ender's Game", + "description": "A young boy is trained to become a military leader in a war against an alien race.", + "author": "Orson Scott Card", + "year": 1985, + }, + { + "name": "Brave New World", + "description": "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", + "author": "Aldous Huxley", + "year": 1932, + }, + { + "name": "The Hitchhiker's Guide to the Galaxy", + "description": "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", + "author": "Douglas Adams", + "year": 1979, + }, + { + "name": "Dune", + "description": "A desert planet is the site of political intrigue and power struggles.", + "author": "Frank Herbert", + "year": 1965, + }, + { + "name": "Foundation", + "description": "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", + "author": "Isaac Asimov", + "year": 1951, + }, + { + "name": "Snow Crash", + "description": "A futuristic world where the internet has evolved into a virtual reality metaverse.", + "author": "Neal Stephenson", + "year": 1992, + }, + { + "name": "Neuromancer", + "description": "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", + "author": "William Gibson", + "year": 1984, + }, + { + "name": "The War of the Worlds", + "description": "A Martian invasion of Earth throws humanity into chaos.", + "author": "H.G. Wells", + "year": 1898, + }, + { + "name": "The Hunger Games", + "description": "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", + "author": "Suzanne Collins", + "year": 2008, + }, + { + "name": "The Andromeda Strain", + "description": "A deadly virus from outer space threatens to wipe out humanity.", + "author": "Michael Crichton", + "year": 1969, + }, + { + "name": "The Left Hand of Darkness", + "description": "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", + "author": "Ursula K. Le Guin", + "year": 1969, + }, + { + "name": "The Three-Body Problem", + "description": "Humans encounter an alien civilization that lives in a dying system.", + "author": "Liu Cixin", + "year": 2008, + }, + } + // @block-end upload-data + + // @block-start upload-points + embeddingModel := "sentence-transformers/all-minilm-l6-v2" + + points := make([]*qdrant.PointStruct, len(documents)) + for idx, doc := range documents { + points[idx] = &qdrant.PointStruct{ + Id: qdrant.NewIDNum(uint64(idx)), + Vectors: qdrant.NewVectorsDocument(&qdrant.Document{ + Text: doc["description"].(string), + Model: embeddingModel, + }), + Payload: qdrant.NewValueMap(doc), + } + } + + client.Upsert(context.Background(), &qdrant.UpsertPoints{ + CollectionName: collectionName, + Points: points, + }) + // @block-end upload-points + + // @block-start query-engine + queryResult, err := client.Query(context.Background(), &qdrant.QueryPoints{ + CollectionName: collectionName, + Query: qdrant.NewQueryDocument(&qdrant.Document{ + Text: "alien invasion", + Model: embeddingModel, + }), + Limit: qdrant.PtrOf(uint64(3)), + }) + + // @hide-start + if err != nil { + panic(err) + } + // @hide-end + + for _, hit := range queryResult { + fmt.Println(hit.Payload, "score:", hit.Score) + } + // @block-end query-engine + + // @block-start create-payload-index + client.CreateFieldIndex(context.Background(), &qdrant.CreateFieldIndexCollection{ + CollectionName: collectionName, + FieldName: "year", + FieldType: qdrant.FieldType_FieldTypeInteger.Enum(), + }) + // @block-end create-payload-index + + // @block-start query-with-filter + queryResultFiltered, err := client.Query(context.Background(), &qdrant.QueryPoints{ + CollectionName: collectionName, + Query: qdrant.NewQueryDocument(&qdrant.Document{ + Text: "alien invasion", + Model: embeddingModel, + }), + Filter: &qdrant.Filter{ + Must: []*qdrant.Condition{ + qdrant.NewRange("year", &qdrant.Range{ + Gte: qdrant.PtrOf(2000.0), + }), + }, + }, + Limit: qdrant.PtrOf(uint64(1)), + }) + + // @hide-start + if err != nil { + panic(err) + } + // @hide-end + + for _, hit := range queryResultFiltered { + fmt.Println(hit.Payload, "score:", hit.Score) + } + // @block-end query-with-filter +} diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/java.java b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/java.java new file mode 100644 index 000000000..1e5fa0025 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/java.java @@ -0,0 +1,214 @@ +package com.example.snippets_amalgamation; + +import static io.qdrant.client.ConditionFactory.range; +import static io.qdrant.client.PointIdFactory.id; +import static io.qdrant.client.QueryFactory.nearest; +import static io.qdrant.client.ValueFactory.value; +import static io.qdrant.client.VectorFactory.vector; +import static io.qdrant.client.VectorsFactory.vectors; + +import io.qdrant.client.QdrantClient; +import io.qdrant.client.QdrantGrpcClient; +import io.qdrant.client.grpc.Collections.Distance; +import io.qdrant.client.grpc.Collections.PayloadSchemaType; +import io.qdrant.client.grpc.Collections.VectorParams; +import io.qdrant.client.grpc.Common.Filter; +import io.qdrant.client.grpc.Common.Range; +import io.qdrant.client.grpc.JsonWithInt.Value; +import io.qdrant.client.grpc.Points.Document; +import io.qdrant.client.grpc.Points.PointStruct; +import io.qdrant.client.grpc.Points.QueryPoints; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +public class Snippet { + + public static void run() throws Exception { + // @hide-start + String QDRANT_URL = ""; + String QDRANT_API_KEY = ""; + // @hide-end + // @block-start client-connection + QdrantClient client = + new QdrantClient( + QdrantGrpcClient.newBuilder(QDRANT_URL, 6334, true) + .withApiKey(QDRANT_API_KEY) + .build()); + // @block-end client-connection + + // @block-start create-collection + String COLLECTION_NAME = "my_books"; + + client.createCollectionAsync(COLLECTION_NAME, + VectorParams.newBuilder().setDistance(Distance.Cosine).setSize(384).build()).get(); + // @block-end create-collection + + // @block-start upload-data + List> payloads = List.of( + Map.of( + "name", value("The Time Machine"), + "description", value("A man travels through time and witnesses the evolution of humanity."), + "author", value("H.G. Wells"), + "year", value(1895)), + Map.of( + "name", value("Ender's Game"), + "description", + value("A young boy is trained to become a military leader in a war against an alien race."), + "author", value("Orson Scott Card"), + "year", value(1985)), + Map.of( + "name", value("Brave New World"), + "description", + value( + "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy."), + "author", value("Aldous Huxley"), + "year", value(1932)), + Map.of( + "name", value("The Hitchhiker's Guide to the Galaxy"), + "description", + value( + "A comedic science fiction series following the misadventures of an unwitting human and his alien friend."), + "author", value("Douglas Adams"), + "year", value(1979)), + Map.of( + "name", value("Dune"), + "description", value("A desert planet is the site of political intrigue and power struggles."), + "author", value("Frank Herbert"), + "year", value(1965)), + Map.of( + "name", value("Foundation"), + "description", + value( + "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse."), + "author", value("Isaac Asimov"), + "year", value(1951)), + Map.of( + "name", value("Snow Crash"), + "description", + value("A futuristic world where the internet has evolved into a virtual reality metaverse."), + "author", value("Neal Stephenson"), + "year", value(1992)), + Map.of( + "name", value("Neuromancer"), + "description", + value( + "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue."), + "author", value("William Gibson"), + "year", value(1984)), + Map.of( + "name", value("The War of the Worlds"), + "description", value("A Martian invasion of Earth throws humanity into chaos."), + "author", value("H.G. Wells"), + "year", value(1898)), + Map.of( + "name", value("The Hunger Games"), + "description", + value("A dystopian society where teenagers are forced to fight to the death in a televised spectacle."), + "author", value("Suzanne Collins"), + "year", value(2008)), + Map.of( + "name", value("The Andromeda Strain"), + "description", value("A deadly virus from outer space threatens to wipe out humanity."), + "author", value("Michael Crichton"), + "year", value(1969)), + Map.of( + "name", value("The Left Hand of Darkness"), + "description", + value( + "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will."), + "author", value("Ursula K. Le Guin"), + "year", value(1969)), + Map.of( + "name", value("The Three-Body Problem"), + "description", value("Humans encounter an alien civilization that lives in a dying system."), + "author", value("Liu Cixin"), + "year", value(2008))); + // @block-end upload-data + + // @block-start upload-points + String EMBEDDING_MODEL = "sentence-transformers/all-minilm-l6-v2"; + + List points = new ArrayList<>(); + + for (int idx = 0; idx < payloads.size(); idx++) { + Map payload = payloads.get(idx); + String description = payload.get("description").getStringValue(); + + PointStruct point = + PointStruct.newBuilder() + .setId(id((long) idx)) + .setVectors( + vectors( + vector( + Document.newBuilder() + .setText(description) + .setModel(EMBEDDING_MODEL) + .build()))) + .putAllPayload(payload) + .build(); + + points.add(point); + } + + client.upsertAsync(COLLECTION_NAME, points).get(); + // @block-end upload-points + + // @block-start query-engine + QueryPoints request = + QueryPoints.newBuilder() + .setCollectionName(COLLECTION_NAME) + .setQuery( + nearest( + Document.newBuilder() + .setText("alien invasion") + .setModel(EMBEDDING_MODEL) + .build())) + .setLimit(3) + .build(); + + var hits = client.queryAsync(request).get(); + + for (var hit : hits) { + System.out.println(hit.getPayloadMap() + " score: " + hit.getScore()); + } + // @block-end query-engine + + // @block-start create-payload-index + client + .createPayloadIndexAsync( + COLLECTION_NAME, + "year", + PayloadSchemaType.Integer, + null, + true, + null, + null) + .get(); + // @block-end create-payload-index + + // @block-start query-with-filter + QueryPoints filteredRequest = + QueryPoints.newBuilder() + .setCollectionName(COLLECTION_NAME) + .setQuery( + nearest( + Document.newBuilder() + .setText("alien invasion") + .setModel(EMBEDDING_MODEL) + .build())) + .setFilter( + Filter.newBuilder() + .addMust(range("year", Range.newBuilder().setGte(2000.0).build())) + .build()) + .setLimit(1) + .build(); + + var filteredHits = client.queryAsync(filteredRequest).get(); + + for (var hit : filteredHits) { + System.out.println(hit.getPayloadMap() + " score: " + hit.getScore()); + } + // @block-end query-with-filter + } +} diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/python.py b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/python.py new file mode 100644 index 000000000..88f564387 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/python.py @@ -0,0 +1,167 @@ +# @hide-start +# mypy: disable-error-code="arg-type" +QDRANT_URL="" +QDRANT_API_KEY="" +# @hide-end +# @block-start client-connection +from qdrant_client import QdrantClient, models + +client = QdrantClient( + url=QDRANT_URL, + api_key=QDRANT_API_KEY, + cloud_inference=True +) +# @block-end client-connection + +# @block-start create-collection +COLLECTION_NAME="my_books" + +client.create_collection( + collection_name=COLLECTION_NAME, + vectors_config=models.VectorParams( + size=384, # Vector size is defined by used model + distance=models.Distance.COSINE, + ), +) +# @block-end create-collection + +# @block-start upload-data +documents = [ + { + "name": "The Time Machine", + "description": "A man travels through time and witnesses the evolution of humanity.", + "author": "H.G. Wells", + "year": 1895, + }, + { + "name": "Ender's Game", + "description": "A young boy is trained to become a military leader in a war against an alien race.", + "author": "Orson Scott Card", + "year": 1985, + }, + { + "name": "Brave New World", + "description": "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", + "author": "Aldous Huxley", + "year": 1932, + }, + { + "name": "The Hitchhiker's Guide to the Galaxy", + "description": "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", + "author": "Douglas Adams", + "year": 1979, + }, + { + "name": "Dune", + "description": "A desert planet is the site of political intrigue and power struggles.", + "author": "Frank Herbert", + "year": 1965, + }, + { + "name": "Foundation", + "description": "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", + "author": "Isaac Asimov", + "year": 1951, + }, + { + "name": "Snow Crash", + "description": "A futuristic world where the internet has evolved into a virtual reality metaverse.", + "author": "Neal Stephenson", + "year": 1992, + }, + { + "name": "Neuromancer", + "description": "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", + "author": "William Gibson", + "year": 1984, + }, + { + "name": "The War of the Worlds", + "description": "A Martian invasion of Earth throws humanity into chaos.", + "author": "H.G. Wells", + "year": 1898, + }, + { + "name": "The Hunger Games", + "description": "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", + "author": "Suzanne Collins", + "year": 2008, + }, + { + "name": "The Andromeda Strain", + "description": "A deadly virus from outer space threatens to wipe out humanity.", + "author": "Michael Crichton", + "year": 1969, + }, + { + "name": "The Left Hand of Darkness", + "description": "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", + "author": "Ursula K. Le Guin", + "year": 1969, + }, + { + "name": "The Three-Body Problem", + "description": "Humans encounter an alien civilization that lives in a dying system.", + "author": "Liu Cixin", + "year": 2008, + }, +] +# @block-end upload-data + +# @block-start upload-points +EMBEDDING_MODEL="sentence-transformers/all-minilm-l6-v2" + +client.upload_points( + collection_name=COLLECTION_NAME, + points=[ + models.PointStruct( + id=idx, + vector=models.Document( + text=doc["description"], + model=EMBEDDING_MODEL + ), + payload=doc + ) + for idx, doc in enumerate(documents) + ], +) +# @block-end upload-points + +# @block-start query-engine +hits = client.query_points( + collection_name=COLLECTION_NAME, + query=models.Document( + text="alien invasion", + model=EMBEDDING_MODEL + ), + limit=3, +).points + +for hit in hits: + print(hit.payload, "score:", hit.score) +# @block-end query-engine + +# @block-start create-payload-index +client.create_payload_index( + collection_name=COLLECTION_NAME, + field_name="year", + field_schema=models.PayloadSchemaType.INTEGER, +) +# @block-end create-payload-index + +# @block-start query-with-filter +hits = client.query_points( + collection_name=COLLECTION_NAME, + query=models.Document( + text="alien invasion", + model=EMBEDDING_MODEL + ), + query_filter=models.Filter( + must=[models.FieldCondition(key="year", range=models.Range(gte=2000))] + ), + limit=1, +).points + +for hit in hits: + print(hit.payload, "score:", hit.score) +# @block-end query-with-filter diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-engine/_description.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-engine/_description.md deleted file mode 100644 index 1f5c96f49..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-engine/_description.md +++ /dev/null @@ -1 +0,0 @@ -This code snippet shows how to query a collection using inference at query time. Instead of providing an explicit query vector, the example uses a `Document` object with query text and a model name. Qdrant generates embeddings from the text and performs a semantic search to find the three most similar books, returning their payloads and similarity scores. diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-engine/python.py b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-engine/python.py deleted file mode 100644 index dff2e8199..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-engine/python.py +++ /dev/null @@ -1,24 +0,0 @@ -# @hide-start -from qdrant_client import QdrantClient, models - -client = QdrantClient( - url="", - api_key="", - cloud_inference=True -) - -COLLECTION_NAME="" -EMBEDDING_MODEL="" -# @hide-end - -hits = client.query_points( - collection_name=COLLECTION_NAME, - query=models.Document( - text="alien invasion", - model=EMBEDDING_MODEL - ), - limit=3, -).points - -for hit in hits: - print(hit.payload, "score:", hit.score) diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-with-filter/_description.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-with-filter/_description.md deleted file mode 100644 index abd0b43ae..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-with-filter/_description.md +++ /dev/null @@ -1 +0,0 @@ -This code snippet demonstrates how to query a collection with a filter. The example uses inference to generate query embeddings and applies a filter to return only books published after the year 2000, limiting the results to the single most relevant match. diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-with-filter/python.py b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-with-filter/python.py deleted file mode 100644 index 1d70e9272..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/query-with-filter/python.py +++ /dev/null @@ -1,27 +0,0 @@ -# @hide-start -from qdrant_client import QdrantClient, models - -client = QdrantClient( - url="", - api_key="", - cloud_inference=True -) - -COLLECTION_NAME="" -EMBEDDING_MODEL="" -# @hide-end - -hits = client.query_points( - collection_name=COLLECTION_NAME, - query=models.Document( - text="alien invasion", - model=EMBEDDING_MODEL - ), - query_filter=models.Filter( - must=[models.FieldCondition(key="year", range=models.Range(gte=2000))] - ), - limit=1, -).points - -for hit in hits: - print(hit.payload, "score:", hit.score) diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/rust.rs b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/rust.rs new file mode 100644 index 000000000..d58420322 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/rust.rs @@ -0,0 +1,125 @@ +use qdrant_client::qdrant::{ + Condition, CreateCollectionBuilder, CreateFieldIndexCollectionBuilder, Distance, Document, + FieldType, Filter, PointStruct, Query, QueryPointsBuilder, Range, UpsertPointsBuilder, VectorParamsBuilder, +}; +use qdrant_client::Qdrant; + +pub async fn main() -> anyhow::Result<()> { + // @hide-start + let QDRANT_URL = ""; + let QDRANT_API_KEY = ""; + // @hide-end + // @block-start client-connection + let client = Qdrant::from_url(QDRANT_URL) + .api_key(QDRANT_API_KEY) + .build()?; + // @block-end client-connection + + // @block-start create-collection + let collection_name = "my_books"; + + client + .create_collection( + CreateCollectionBuilder::new(collection_name) + .vectors_config(VectorParamsBuilder::new(384, Distance::Cosine)), // Vector size is defined by used model + ) + .await?; + // @block-end create-collection + + // @block-start upload-data + let documents = [ + ("The Time Machine", "A man travels through time and witnesses the evolution of humanity.", "H.G. Wells", 1895), + ("Ender's Game", "A young boy is trained to become a military leader in a war against an alien race.", "Orson Scott Card", 1985), + ("Brave New World", "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", "Aldous Huxley", 1932), + ("The Hitchhiker's Guide to the Galaxy", "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", "Douglas Adams", 1979), + ("Dune", "A desert planet is the site of political intrigue and power struggles.", "Frank Herbert", 1965), + ("Foundation", "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", "Isaac Asimov", 1951), + ("Snow Crash", "A futuristic world where the internet has evolved into a virtual reality metaverse.", "Neal Stephenson", 1992), + ("Neuromancer", "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", "William Gibson", 1984), + ("The War of the Worlds", "A Martian invasion of Earth throws humanity into chaos.", "H.G. Wells", 1898), + ("The Hunger Games", "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", "Suzanne Collins", 2008), + ("The Andromeda Strain", "A deadly virus from outer space threatens to wipe out humanity.", "Michael Crichton", 1969), + ("The Left Hand of Darkness", "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", "Ursula K. Le Guin", 1969), + ("The Three-Body Problem", "Humans encounter an alien civilization that lives in a dying system.", "Liu Cixin", 2008), + ]; + // @block-end upload-data + + // @block-start upload-points + let embedding_model = "sentence-transformers/all-minilm-l6-v2"; + + let points: Vec = documents + .iter() + .enumerate() + .map(|(idx, (name, description, author, year))| { + PointStruct::new( + idx as u64, + Document::new(*description, embedding_model), + [ + ("name", (*name).into()), + ("description", (*description).into()), + ("author", (*author).into()), + ("year", (*year).into()), + ], + ) + }) + .collect(); + + client + .upsert_points(UpsertPointsBuilder::new(collection_name, points)) + .await?; + // @block-end upload-points + + // @block-start query-engine + let query_result = client + .query( + QueryPointsBuilder::new(collection_name) + .query(Query::new_nearest(Document::new( + "alien invasion", + embedding_model, + ))) + .limit(3) + .with_payload(true), + ) + .await?; + + for hit in query_result.result { + println!("{:?} score: {}", hit.payload, hit.score); + } + // @block-end query-engine + + // @block-start create-payload-index + client + .create_field_index( + CreateFieldIndexCollectionBuilder::new(collection_name, "year", FieldType::Integer) + .wait(true), + ) + .await?; + // @block-end create-payload-index + + // @block-start query-with-filter + let query_result_filtered = client + .query( + QueryPointsBuilder::new(collection_name) + .query(Query::new_nearest(Document::new( + "alien invasion", + embedding_model, + ))) + .filter(Filter::must([Condition::range( + "year", + Range { + gte: Some(2000.0), + ..Default::default() + }, + )])) + .limit(1) + .with_payload(true), + ) + .await?; + + for hit in query_result_filtered.result { + println!("{:?} score: {}", hit.payload, hit.score); + } + // @block-end query-with-filter + + Ok(()) +} diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/typescript.ts b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/typescript.ts new file mode 100644 index 000000000..675678ac9 --- /dev/null +++ b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/typescript.ts @@ -0,0 +1,101 @@ +import { QdrantClient } from "@qdrant/js-client-rest"; + +// @hide-start +const QDRANT_URL = ""; +const QDRANT_API_KEY = ""; +// @hide-end +// @block-start client-connection +const client = new QdrantClient({ + url: QDRANT_URL, + apiKey: QDRANT_API_KEY, +}); +// @block-end client-connection + +// @block-start create-collection +const collectionName = "my_books"; + +await client.createCollection(collectionName, { + vectors: { + size: 384, // Vector size is defined by used model + distance: "Cosine", + }, +}); +// @block-end create-collection + +// @block-start upload-data +const documents = [ + { name: "The Time Machine", description: "A man travels through time and witnesses the evolution of humanity.", author: "H.G. Wells", year: 1895 }, + { name: "Ender's Game", description: "A young boy is trained to become a military leader in a war against an alien race.", author: "Orson Scott Card", year: 1985 }, + { name: "Brave New World", description: "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", author: "Aldous Huxley", year: 1932 }, + { name: "The Hitchhiker's Guide to the Galaxy", description: "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", author: "Douglas Adams", year: 1979 }, + { name: "Dune", description: "A desert planet is the site of political intrigue and power struggles.", author: "Frank Herbert", year: 1965 }, + { name: "Foundation", description: "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", author: "Isaac Asimov", year: 1951 }, + { name: "Snow Crash", description: "A futuristic world where the internet has evolved into a virtual reality metaverse.", author: "Neal Stephenson", year: 1992 }, + { name: "Neuromancer", description: "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", author: "William Gibson", year: 1984 }, + { name: "The War of the Worlds", description: "A Martian invasion of Earth throws humanity into chaos.", author: "H.G. Wells", year: 1898 }, + { name: "The Hunger Games", description: "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", author: "Suzanne Collins", year: 2008 }, + { name: "The Andromeda Strain", description: "A deadly virus from outer space threatens to wipe out humanity.", author: "Michael Crichton", year: 1969 }, + { name: "The Left Hand of Darkness", description: "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", author: "Ursula K. Le Guin", year: 1969 }, + { name: "The Three-Body Problem", description: "Humans encounter an alien civilization that lives in a dying system.", author: "Liu Cixin", year: 2008 }, +]; +// @block-end upload-data + +// @block-start upload-points +const embeddingModel = "sentence-transformers/all-minilm-l6-v2"; + +const points = documents.map((doc, idx) => ({ + id: idx, + vector: { + text: doc.description, + model: embeddingModel, + }, + payload: doc, +})); + +await client.upsert(collectionName, { points }); +// @block-end upload-points + +// @block-start query-engine +const queryResult = await client.query(collectionName, { + query: { + text: "alien invasion", + model: embeddingModel, + }, + limit: 3, +}); + +for (const hit of queryResult.points) { + console.log(hit.payload, "score:", hit.score); +} +// @block-end query-engine + +// @block-start create-payload-index +await client.createPayloadIndex(collectionName, { + field_name: "year", + field_schema: "integer", +}); +// @block-end create-payload-index + +// @block-start query-with-filter +const queryResultFiltered = await client.query(collectionName, { + query: { + text: "alien invasion", + model: embeddingModel, + }, + filter: { + must: [ + { + key: "year", + range: { + gte: 2000, + }, + }, + ], + }, + limit: 1, +}); + +for (const hit of queryResultFiltered.points) { + console.log(hit.payload, "score:", hit.score); +} +// @block-end query-with-filter diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-data/_description.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-data/_description.md deleted file mode 100644 index d5531e6f6..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-data/_description.md +++ /dev/null @@ -1 +0,0 @@ -This code snippet defines a dataset of science fiction books. Each book entry contains a name, description, author, and publication year. This dataset will be uploaded to the Qdrant collection for semantic search. diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/_description.md b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/_description.md deleted file mode 100644 index 83c08c122..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/_description.md +++ /dev/null @@ -1 +0,0 @@ -This code snippet demonstrates how to upload points to a collection using inference at ingest time. Instead of providing explicit vectors, the example uses a `Document` object with the book description and a model name. Qdrant generates embeddings from the text using the specified model and stores the resulting vectors along with the book's metadata as payload. diff --git a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/python.py b/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/python.py deleted file mode 100644 index 40c145741..000000000 --- a/qdrant-landing/content/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/python.py +++ /dev/null @@ -1,37 +0,0 @@ -# @hide-start -# mypy: disable-error-code="arg-type" -from qdrant_client import QdrantClient, models - -client = QdrantClient( - url="", - api_key="", - cloud_inference=True -) - -COLLECTION_NAME="" -documents = documents = [ - { - "name": "", - "description": "", - "author": "", - "year": 1895, - }] -# @hide-end - -# Define the embedding model used by Cloud Inference -EMBEDDING_MODEL="sentence-transformers/all-minilm-l6-v2" - -client.upload_points( - collection_name=COLLECTION_NAME, - points=[ - models.PointStruct( - id=idx, - vector=models.Document( - text=doc["description"], - model=EMBEDDING_MODEL # Cloud Inference generates embeddings with this model - ), - payload=doc - ) - for idx, doc in enumerate(documents) - ], -) \ No newline at end of file diff --git a/qdrant-landing/content/documentation/tutorials-basics/search-beginners.md b/qdrant-landing/content/documentation/tutorials-basics/search-beginners.md index d689c9c37..efdc8b164 100644 --- a/qdrant-landing/content/documentation/tutorials-basics/search-beginners.md +++ b/qdrant-landing/content/documentation/tutorials-basics/search-beginners.md @@ -21,7 +21,7 @@ aliases: If you are new to vector search engines, this tutorial is for you. In 5 minutes you will build a semantic search engine for science fiction books. After you set it up, you will ask the engine about an impending alien threat. Your creation will recommend books as preparation for a potential space attack. -Before you begin, you need to have a [recent version of Python](https://www.python.org/downloads/) installed. If you don't know how to run this code in a virtual environment, follow the Python documentation for [creating virtual environments](https://docs.python.org/3/tutorial/venv.html#creating-virtual-environments) first. Alternatively, you can use [this Google Colab notebook](https://githubtocolab.com/qdrant/examples/blob/master/semantic-search-in-5-minutes/semantic_search_in_5_minutes.ipynb). +If you are using Python, you can use [this Google Colab notebook](https://githubtocolab.com/qdrant/examples/blob/master/semantic-search-in-5-minutes/semantic_search_in_5_minutes.ipynb). ## 1. Create a Qdrant Cluster @@ -36,15 +36,15 @@ If you do not already have a Qdrant cluster, follow these steps to create one: ## 2. Set up a Client Connection -First, install the Qdrant Client for Python. This library allows you to interact with Qdrant from Python code. +First, install the Qdrant Client for your preferred programming language: -```bash -pip install qdrant-client -``` +{{< code-snippet path="/documentation/headless/snippets/install-client/" >}} + +This library allows you to interact with Qdrant from code. Next, create a client connection to your Qdrant cluster using the endpoint and API key. -{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/client-connection/" >}} +{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/" block="client-connection" >}} Replace `QDRANT_URL` and `QDRANT_API_KEY` with the cluster endpoint and API key you obtained in the previous step. The `cloud_inference=True` parameter enables Qdrant Cloud's [inference](/documentation/concepts/inference/) capabilities, allowing the cluster to generate vector embeddings without the need to manage your own embedding infrastructure. @@ -52,7 +52,7 @@ Replace `QDRANT_URL` and `QDRANT_API_KEY` with the cluster endpoint and API key All data in Qdrant is organized within [collections](/documentation/concepts/collections/). Since you're storing books, let's create a collection named `my_books`. -{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/create-collection/" >}} +{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/" block="create-collection" >}} - The `size` parameter defines the dimensionality of the vectors for the collection. 384 corresponds to the output dimensionality of the embedding model used in this tutorial. - The `distance` parameter specifies the function used to measure the distance between two points. @@ -61,11 +61,11 @@ All data in Qdrant is organized within [collections](/documentation/concepts/col The dataset consists of a list of science fiction books. Each entry has a name, author, publication year, and short description. -{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/upload-data/" >}} +{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/" block="upload-data" >}} Store each book as a [point](/documentation/concepts/points/) in the `my_books` collection, with each point consisting of a [unique ID](/documentation/concepts/points/#point-ids), a [vector](/documentation/concepts/vectors/) generated from the description, and a [payload](/documentation/concepts/payload/) containing the book's metadata: -{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/upload-points/" >}} +{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/" block="upload-points" >}} This code tells Qdrant Cloud to use the `sentence-transformers/all-minilm-l6-v2` embedding model to generate vector embeddings from the book descriptions. This is one of the free models available on Qdrant Cloud. For a list of the available free and paid models, refer to the Inference tab of the Cluster Detail page in the Qdrant Cloud Console. @@ -73,7 +73,7 @@ This code tells Qdrant Cloud to use the `sentence-transformers/all-minilm-l6-v2` Now that the data is stored in Qdrant, you can query it and receive semantically relevant results. -{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/query-engine/" >}} +{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/" block="query-engine" >}} This query uses the same embedding model to generate a vector for the query "alien invasion". The search engine then looks for the three most similar vectors in the collection and returns their payloads and similarity scores. @@ -93,13 +93,13 @@ How about the most recent book from the early 2000s? Qdrant allows you to narrow Before filtering on a payload field, create a [payload index](/documentation/concepts/indexing/#payload-index) for that field: -{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/create-payload-index/" >}} +{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/" block="create-payload-index" >}} In a production environment, create payload indexes before uploading data to get the maximum benefit from indexing. Now you can apply a filter to the query: -{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/query-with-filter/" >}} +{{< code-snippet path="/documentation/headless/snippets/tutorial-semantic-search-101/" block="query-with-filter" >}} **Response:** diff --git a/qdrant-landing/themes/qdrant-2024/layouts/shortcodes/code-snippet.html b/qdrant-landing/themes/qdrant-2024/layouts/shortcodes/code-snippet.html index e5c204716..8e0e4b2db 100644 --- a/qdrant-landing/themes/qdrant-2024/layouts/shortcodes/code-snippet.html +++ b/qdrant-landing/themes/qdrant-2024/layouts/shortcodes/code-snippet.html @@ -12,6 +12,7 @@ {{ $path := .Get "path" }} {{ $order := .Get "order" }} +{{ $block := .Get "block" }} @@ -53,7 +54,7 @@ {{ $files := dict }} -{{ range $dir := slice (printf "content/%s" $path) (printf "content/%s/generated" $path) }} +{{ range $dir := slice (printf "content/%s" $path) (printf "content/%s/generated/%s" $path $block) }} {{ if not (fileExists $dir) }} {{ continue }} {{ end }}