Google Summer of Code 2023 - Web UI for Visualization and Exploration (#250)

* Google Summer of Code 2023 - Web UI for Visualization and Exploration

* Key Learning Points highlight key

* proofreading and editing

Excellent work @kartik-gupta-ij, we are very proud of you!.

* Page Images added

* upd preview

* upd order and date

---------

Co-authored-by: David Myriel <davidmyriel@gmail.com>
Co-authored-by: generall <andrey@vasnetsov.com>

qdrant-landing/content/articles/web-ui-gsoc.md

@@ -0,0 +1,130 @@
+---
+title:  Google Summer of Code 2023 - Web UI for Visualization and Exploration
+short_description: Gsoc'23 Web UI for Visualization and Exploration
+description: My journey as a Google Summer of Code 2023 student working on the "Web UI for Visualization and Exploration" project for Qdrant.
+preview_dir: /articles_data/web-ui-gsoc/preview
+small_preview_image: /articles_data/web-ui-gsoc/icon.svg
+social_preview_image: /articles_data/web-ui-gsoc/preview/social_preview.jpg
+weight: -20
+author: Kartik Gupta
+author_link: https://kartik-gupta-ij.vercel.app/
+date: 2023-08-28T08:00:00+03:00
+draft: false
+keywords: 
+
+    - vector reduction
+    - console 
+    - gsoc'23 
+    - vector similarity
+    - exploration
+    - recommendation
+---
+
+
+
+## Introduction
+
+Hello everyone! My name is Kartik Gupta, and I am thrilled to share my coding journey as part of the Google Summer of Code 2023 program. This summer, I had the incredible opportunity to work on an exciting project titled "Web UI for Visualization and Exploration" for Qdrant, a vector search engine. In this article, I will take you through my experience, challenges, and achievements during this enriching coding journey.
+
+## Project Overview
+
+Qdrant is a powerful vector search engine widely used for similarity search and clustering. However, it lacked a user-friendly web-based UI for data visualization and exploration. My project aimed to bridge this gap by developing a web-based user interface that allows users to easily interact with and explore their vector data.
+
+## Milestones and Achievements
+
+The project was divided into six milestones, each focusing on a specific aspect of the web UI development. Let's go through each of them and my achievements during the coding period.
+
+**1. Designing a friendly UI on Figma** 
+
+I started by designing the user interface on Figma, ensuring it was easy to use, visually appealing, and responsive on different devices. I focused on usability and accessibility to create a seamless user experience. ( [Figma Design](https://www.figma.com/file/z54cAcOErNjlVBsZ1DrXyD/Qdant?type=design&node-id=0-1&mode=design&t=Pu22zO2AMFuGhklG-0))
+    
+**2. Building the layout** 
+
+The layout route served as a landing page with an overview of the application's features and navigation links to other routes.
+
+**3. Creating a view collection route** 
+
+This route enabled users to view a list of collections available in the application. Users could click on a collection to see more details, including the data and vectors associated with it.
+
+{{< figure src=/articles_data/web-ui-gsoc/collections-page.png caption="Collection Page" alt="Collection Page" >}}
+
+**4. Developing a data page with "find similar" functionality** 
+
+I implemented a data page where users could search for data and find similar data using a recommendation API. The recommendation API suggested similar data based on the Data's selected ID, providing valuable insights.
+
+{{< figure src=/articles_data/web-ui-gsoc/points-page.png caption="Points Page" alt="Points Page" >}}
+    
+**5. Developing query editor page libraries** 
+
+This milestone involved creating a query editor page that allowed users to write queries in a custom language. The editor provided syntax highlighting, autocomplete, and error-checking features for a seamless query writing experience.
+
+{{< figure src=/articles_data/web-ui-gsoc/console-page.png caption="Query Editor Page" alt="Query Editor Page" >}}
+    
+**6. Developing a route for visualizing vector data points**
+
+This is done by the reduction of n-dimensional vector in 2-D points and they are displayed with their respective payloads.
+
+{{< figure src=/articles_data/web-ui-gsoc/visualization-page.png caption="Vector Visuliztion Page" alt="visualization-page" >}}
+
+## Challenges and Learning
+
+Throughout the project, I encountered a series of challenges that stretched my engineering capabilities and provided unique growth opportunities. From mastering new libraries and technologies to ensuring the user interface (UI) was both visually appealing and user-friendly, every obstacle became a stepping stone toward enhancing my skills as a developer. However, each challenge provided an opportunity to learn and grow as a developer. I acquired valuable experience in vector search and dimension reduction techniques.
+
+The most significant learning for me was the importance of effective project management. Setting realistic timelines, collaborating with mentors, and staying proactive with feedback allowed me to complete the milestones efficiently.
+
+### Technical Learning and Skill Development
+
+One of the most significant aspects of this journey was diving into the intricate world of vector search and dimension reduction techniques. These areas, previously unfamiliar to me, required rigorous study and exploration. Learning how to process vast amounts of data efficiently and extract meaningful insights through these techniques was both challenging and rewarding.
+
+### Effective Project Management
+
+Undoubtedly, the most impactful lesson was the art of effective project management. I quickly grasped the importance of setting realistic timelines and goals. Collaborating closely with mentors and maintaining proactive communication proved indispensable. This approach enabled me to navigate the complex development process and successfully achieve the project's milestones.
+
+### Overcoming Technical Challenges
+
+#### Autocomplete Feature in Console
+
+One particularly intriguing challenge emerged while working on the autocomplete feature within the console. Finding a solution was proving elusive until a breakthrough came from an unexpected direction. My mentor, Andrey, proposed creating a separate module that could support autocomplete based on OpenAPI for our custom language. This ingenious approach not only resolved the issue but also showcased the power of collaborative problem-solving.
+
+#### Optimization with Web Workers
+
+The high-processing demands of vector reduction posed another significant challenge. Initially, this task was straining browsers and causing performance issues. The solution materialized in the form of web workers—an independent processing instance that alleviated the strain on browsers. However, a new question arose: how to terminate these workers effectively? With invaluable insights from my mentor, I gained a deeper understanding of web worker dynamics and successfully tackled this challenge.
+
+#### Console Integration Complexity
+
+Integrating the console interaction into the application presented multifaceted challenges. Crafting a custom language in Monaco, parsing text to make API requests, and synchronizing the entire process demanded meticulous attention to detail. Overcoming these hurdles was a testament to the complexity of real-world engineering endeavours.
+
+#### Codelens Multiplicity Issue
+
+An unexpected issue cropped up during the development process: the codelen (run button) registered multiple times, leading to undesired behaviour. This hiccup underscored the importance of thorough testing and debugging, even in seemingly straightforward features.
+
+### Key Learning Points
+
+Amidst these challenges, I garnered valuable insights that have significantly enriched my engineering prowess:
+
+**Vector Reduction Techniques**: Navigating the realm of vector reduction techniques provided a deep understanding of how to process and interpret data efficiently. This knowledge opens up new avenues for developing data-driven applications in the future.
+
+**Web Workers Efficiency**: Mastering the intricacies of web workers not only resolved performance concerns but also expanded my repertoire of optimization strategies. This newfound proficiency will undoubtedly find relevance in various future projects.
+
+**Monaco Editor and UI Frameworks**: Working extensively with the Monaco Editor, Material-UI (MUI), and Vite enriched my familiarity with these essential tools. I honed my skills in integrating complex UI components seamlessly into applications.
+
+## Areas for Improvement and Future Enhancements
+
+While reflecting on this transformative journey, I recognize several areas that offer room for improvement and future enhancements:
+
+1. Enhanced Autocomplete: Further refining the autocomplete feature to support key-value suggestions in JSON structures could greatly enhance the user experience.
+    
+2. Error Detection in Console: Integrating the console's error checker with OpenAPI could enhance its accuracy in identifying errors and offering precise suggestions for improvement.
+    
+3. Expanded Vector Visualization: Exploring additional visualization methods and optimizing their performance could elevate the utility of the vector visualization route.
+    
+
+## Conclusion
+
+Participating in the Google Summer of Code 2023 and working on the "Web UI for Visualization and Exploration" project has been an immensely rewarding experience. I am grateful for the opportunity to contribute to Qdrant and develop a user-friendly interface for vector data exploration.
+
+I want to express my gratitude to my mentors and the entire Qdrant community for their support and guidance throughout this journey. This experience has not only improved my coding skills but also instilled a deeper passion for web development and data analysis.
+
+As my coding journey continues beyond this project, I look forward to applying the knowledge and experience gained here to future endeavours. I am excited to see how Qdrant evolves with the newly developed web UI and how it positively impacts users worldwide.
+
+Thank you for joining me on this coding adventure, and I hope to share more exciting projects in the future! Happy coding!

qdrant-landing/static/articles_data/web-ui-gsoc/icon.svg

@@ -0,0 +1,75 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<svg
+   version="1.0"
+   width="512.000000pt"
+   height="512.000000pt"
+   viewBox="0 0 512.000000 512.000000"
+   preserveAspectRatio="xMidYMid meet"
+   id="svg20"
+   sodipodi:docname="servers.svg"
+   inkscape:version="1.1.2 (0a00cf5339, 2022-02-04)"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:svg="http://www.w3.org/2000/svg">
+  <defs
+     id="defs24" />
+  <sodipodi:namedview
+     id="namedview22"
+     pagecolor="#ffffff"
+     bordercolor="#999999"
+     borderopacity="1"
+     inkscape:pageshadow="0"
+     inkscape:pageopacity="0"
+     inkscape:pagecheckerboard="true"
+     inkscape:document-units="pt"
+     showgrid="false"
+     inkscape:zoom="0.47556786"
+     inkscape:cx="-90.418221"
+     inkscape:cy="424.75536"
+     inkscape:window-width="2560"
+     inkscape:window-height="1011"
+     inkscape:window-x="0"
+     inkscape:window-y="32"
+     inkscape:window-maximized="1"
+     inkscape:current-layer="g18" />
+  <g
+     transform="translate(0.000000,512.000000) scale(0.100000,-0.100000)"
+     fill="#000000"
+     stroke="none"
+     id="g18"
+     style="fill:#ffffff">
+    <path
+       d="M1110 5113 c-51 -8 -126 -36 -178 -65 -109 -61 -139 -103 -492 -668 -358 -574 -410 -666 -428 -756 -19 -90 -16 -574 3 -649 24 -94 71 -179 135 -247 l59 -62 -60 -63 c-68 -73 -113 -155 -134 -245 -21 -91 -21 -579 0 -663 24 -94 71 -179 135 -247 l59 -62 -60 -63 c-32 -35 -73 -91 -89 -123 -53 -105 -60 -157 -60 -455 0 -302 8 -355 63 -459 41 -78 145 -182 223 -223 119 -63 113 -63 927 -63 821 0 784 -3 813 67 19 44 11 83 -22 114 l-26 24 -757 5 -756 5 -67 33 c-76 37 -120 83 -157 162 -25 53 -26 62 -29 295 -4 255 3 330 39 399 30 59 95 121 153 148 l51 23 951 5 c825 4 953 7 970 20 35 27 43 45 44 87 0 33 -6 48 -30 72 l-31 30 -947 3 -947 3 -67 33 c-76 37 -120 83 -157 162 -25 53 -26 62 -29 295 -4 255 3 330 39 399 30 59 95 121 153 148 l51 23 2105 0 2105 0 51 -23 c58 -27 123 -89 153 -148 36 -69 43 -143 39 -394 -4 -212 -6 -241 -26 -299 -28 -77 -18 -121 32 -152 85 -52 151 1 191 156 21 83 22 571 0 663 -21 90 -66 172 -134 245 l-60 63 59 62 c32 34 73 90 89 125 55 113 61 155 61 449 0 287 -6 336 -50 434 -24 54 -611 1007 -712 1156 -44 66 -126 136 -201 172 -121 59 -43 56 -1597 55 -784 -1 -1436 -4 -1450 -6z m2940 -234 c30 -15 70 -40 88 -55 18 -16 136 -195 263 -399 l230 -370 -1036 -3 c-569 -1 -1501 -1 -2070 0 l-1036 3 230 370 c127 204 245 383 263 398 44 38 100 66 151 77 23 4 677 7 1452 7 l1410 -2 55 -26z m666 -1067 c58 -27 123 -89 153 -148 36 -69 43 -144 39 -399 l-3 -240 -33 -67 c-37 -76 -82 -120 -162 -157 l-55 -26 -2095 0 -2095 0 -55 26 c-80 37 -125 81 -162 157 l-33 67 -3 240 c-4 255 3 330 39 399 25 50 83 112 124 134 81 42 12 41 2200 39 l2090 -2 51 -23z"
+       id="path2"
+       style="fill:#ffffff" />
+    <path
+       d="M680 3616 c-71 -16 -107 -35 -157 -86 -68 -67 -88 -119 -88 -225 0 -78 4 -97 27 -140 32 -60 87 -114 148 -146 37 -20 60 -24 135 -24 106 0 158 20 225 88 51 51 75 96 88 167 41 222 -157 414 -378 366z m139 -237 c82 -81 -15 -218 -118 -165 -67 35 -78 122 -20 170 44 38 98 36 138 -5z"
+       id="path4"
+       style="fill:#ffffff" />
+    <path
+       d="M2645 3623 c-88 -23 -111 -140 -37 -190 26 -17 71 -18 912 -18 841 0 886 1 912 18 58 39 58 135 0 174 -26 17 -71 18 -897 19 -478 1 -879 0 -890 -3z"
+       id="path6"
+       style="fill:#ffffff" />
+    <path
+       d="M2618 3184 c-65 -35 -73 -121 -17 -168 l31 -26 888 0 888 0 31 26 c57 48 47 135 -19 169 -43 22 -1761 22 -1802 -1z"
+       id="path8"
+       style="fill:#ffffff" />
+    <path
+       d="M680 2336 c-71 -16 -107 -35 -157 -86 -68 -67 -88 -119 -88 -225 0 -78 4 -97 27 -140 32 -60 87 -114 148 -146 37 -20 60 -24 135 -24 106 0 158 20 225 88 51 51 75 96 88 167 41 222 -157 414 -378 366z m139 -237 c82 -81 -15 -218 -118 -165 -67 35 -78 122 -20 170 44 38 98 36 138 -5z"
+       id="path10"
+       style="fill:#ffffff" />
+    <path
+       d="M2645 2343 c-88 -23 -111 -140 -37 -190 26 -17 71 -18 912 -18 841 0 886 1 912 18 58 39 58 135 0 174 -26 17 -71 18 -897 19 -478 1 -879 0 -890 -3z"
+       id="path12"
+       style="fill:#ffffff" />
+    <path
+       d="M3405 1914 c-11 -2 -45 -9 -75 -15 -284 -54 -537 -249 -668 -514 -47 -96 -59 -130 -81 -242 l-18 -86 -54 -17 c-82 -26 -132 -54 -197 -115 -102 -94 -155 -196 -172 -329 -32 -250 123 -488 370 -572 52 -18 104 -19 1020 -19 903 0 969 1 1030 18 261 73 465 278 536 537 28 99 25 281 -4 385 -72 249 -257 437 -507 517 -35 11 -105 22 -167 25 -105 6 -107 6 -129 37 -154 211 -386 349 -643 385 -74 11 -199 13 -241 5z m283 -230 c134 -28 294 -125 390 -235 16 -19 47 -62 70 -95 23 -34 49 -65 59 -70 10 -4 77 -9 148 -9 104 -2 141 -6 187 -23 243 -89 397 -336 358 -576 -21 -135 -71 -228 -171 -321 -65 -59 -120 -91 -209 -121 -51 -17 -112 -19 -945 -22 -921 -3 -979 -1 -1054 38 -51 26 -115 92 -143 148 -19 39 -23 63 -23 137 0 78 4 97 27 140 52 98 136 156 253 175 117 19 129 33 144 175 19 185 90 335 215 461 186 185 425 254 694 198z"
+       id="path14"
+       style="fill:#ffffff" />
+    <path
+       d="M680 1056 c-71 -16 -107 -35 -157 -86 -68 -67 -88 -119 -88 -225 0 -78 4 -97 27 -140 32 -60 87 -114 148 -146 37 -20 60 -24 135 -24 106 0 158 20 225 88 51 51 75 96 88 167 41 222 -157 414 -378 366z m139 -237 c82 -81 -15 -218 -118 -165 -67 35 -78 122 -20 170 44 38 98 36 138 -5z"
+       id="path16"
+       style="fill:#ffffff" />
+  </g>
+</svg>