An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Related tags

Image Processingexecutor-3d-encoder
Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

  • PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
  • PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

  • PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
  • PointConv: Deep Convolutional Networks on 3D Point Clouds
Comments
  • which version should the `finetuner` module be?

    which version should the `finetuner` module be?

    I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

    from finetuner.tuner.callback import BestModelCheckpoint
from finetuner.tuner.pytorch.losses import TripletLoss
from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

    It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

    Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.

    opened by elenacliu 4
  • fix: fix previous structure

    fix: fix previous structure

    I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

    opened by elenacliu 2
  • Add dataset, preprocess and pointnet2

    Add dataset, preprocess and pointnet2

    Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

    @numb3r3 @jemmyshin

    opened by elenacliu 1
  • In `farthest_point_sample` function, the first centroid sampled is not random.

    In `farthest_point_sample` function, the first centroid sampled is not random.

    I'm reading your codes.

    https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

    The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

    opened by elenacliu 1
  • Provide diffusion encoder

    Provide diffusion encoder

    https://github.com/nmwsharp/diffusion-net

    I'm unfamiliar with the jina ecosystem. So this is my first issue.

    Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

    opened by fire 2
Releases(v0.3.0)
Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
GitHub Repository
ScreenTeX is a tool that grabs all text when taking a screenshot rather than getting an image.

The ScreenTeX project By: Seanpm2001 / ScreenTeX, Et; Al. Top README.md Read this article in a different language 🌐 List of languages Sorted by: A-Z

Sean P. Myrick V19.1.7.2 3 Oct 25, 2022
QR-Generator - An awesome QR Generator to create or customize your QR's

QR Generator An awesome QR Generator to create or customize your QR's! Table of

Tristán 1 Jan 28, 2022
Qt based ebook reader

Qt based ebook reader Currently supports: pdf epub djvu fb2 mobi azw / azw3 / azw4 cbr / cbz md Contribute Paypal Bitcoin: 17jaxj26vFJNqQ2hEVerbBV5fpT

1.4k Dec 26, 2022
Create QR Code for link using Python

Quick Response QR is short and named for a quick read from a cell phone. Used to view information from transitory media and put it on your cell phone.

Coding Taggers 1 Jan 09, 2022
Image enhancing model for making a blurred image to be somehow clearer than before

This is a very small prject which helps in enhancing the images by taking a Input images. This project has many features like detcting the faces and enhaning the faces itself and also a feature which

3 Dec 03, 2021
An API which would colorize a black and white image

Image Colorization API Machine Learning Model used- https://github.com/richzhang/colorization/tree/caffe Paper - https://arxiv.org/abs/1603.08511 Step

Neelesh Ranjan Jha 4 Nov 23, 2021
A minimal, standalone viewer for 3D animations stored as stop-motion sequences of individual .obj mesh files.

ObjSequenceViewer V0.5 A minimal, standalone viewer for 3D animations stored as stop-motion sequences of individual .obj mesh files. Installation: pip

csmailis 2 Aug 04, 2022
Blender addon to generate better building models from satellite imagery.

Blender addon to generate better building models from satellite imagery.

Ivan Ereshchenko 24 Apr 14, 2022
This piece of code is a User Welcomer with Image Manipulation using Python and Pillow (PIL).

This piece of code is a User Welcomer with Image Manipulation using Python and Pillow (PIL).

Bero 4 Jan 11, 2022
Console images in 48 colors, 216 colors and full rgb

console_images Console images in 48 colors, 216 colors and full rgb Full RGB 216 colors 48 colors If it does not work maybe you should change color_fu

Урядов Алексей 5 Oct 11, 2022
Dynamic image server for web and print

Quru Image Server - dynamic imaging for web and print QIS is a high performance web server for creating and delivering dynamic images. It is ideal for

Quru 84 Jan 03, 2023
Hacking github graph with a easy python script

Hacking-Github-Graph Hacking github graph with a easy python script Requirements git latest version installed. A text editor (eg: vs code, sublime tex

SENPAI LEGEND 1 Nov 01, 2021
Semi-hash-based Image Generator

pixel-planet Semi-hash-based Image Generator Utilizable for NFTs Generation Process Input is salted and hashed Colors (background, planet, stars) are

Bill Ni 3 Sep 01, 2022
An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

3D Mesh Encoder An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings o

Jina AI 11 Dec 14, 2022
A warping based image translation model focusing on upper body synthesis.

Pose2Img Upper body image synthesis from skeleton(Keypoints). Sub module in the ICCV-2021 paper "Speech Drives Templates: Co-Speech Gesture Synthesis

zhiyh 15 Nov 10, 2022
QR Code Generator

In this project, we'll be using some libraries to instantly generate authentic QR Codes and export them in various formats

Hassan Shahzad 3 Jun 02, 2022
Python script to generate vector graphics of an oriented lattice unit cell

unitcell Python script to generate vector graphics of an oriented lattice unit cell Examples unitcell --type hexagonal --eulers 12 23 34 --axes --crys

Philip Eisenlohr 2 Dec 10, 2021
API to help generating QR-code for ZATCA's e-invoice known as Fatoora with any programming language

You can try it @ api-fatoora api-fatoora API to help generating QR-code for ZATCA's e-invoice known as Fatoora with any programming language Disclaime

نافع الهلالي 12 Oct 05, 2022
Computer art based on joining transparent images

Computer Art There is no must in art because art is free. Introduction The following tutorial exaplains how to generate computer art based on a series

Computer Art 12 Jul 30, 2022
A Icon Maker GUI Made - Convert your image into icon ( .ico format ).

Icon-Maker-GUI A Icon Maker GUI Made Using Python 3.9.0 . It will take any image and convert it to ICO file, for web site favicon or Windows applicati

Insanecodes 12 Dec 15, 2021