datasets package¶

Submodules¶

datasets.CUB module¶

class datasets.CUB.CUB(data_dir='data_dir', verbose=True)¶: Bases: datasets.bases.BaseImageDataset

datasets.NAbirds module¶

class datasets.NAbirds.NAbirds(data_dir='data_dir', verbose=True)¶: Bases: datasets.bases.BaseImageDataset

datasets.UnifiedLoader module¶

class datasets.UnifiedLoader.UnifiedLoader(dataset_name='Aircraft', data_dir='data_dir', verbose=True)¶

Bases: datasets.bases.BaseImageDataset

This must be self set

can be revised in the next version

verbose: print additional messages

datasets.bases module¶

class datasets.bases.BaseDataset¶

Bases: object

Base class of reading all dataset

get_imagedata_info(data)¶

get_imagedata_info_discrete(data)¶

print_dataset_statistics()¶

class datasets.bases.BaseImageDataset¶

Bases: datasets.bases.BaseDataset

Base class of Imagereading dataset

print_dataset_statistics(train, val)¶

class datasets.bases.ImageDataset(dataset, transform=None)¶: Bases: torch.utils.data.dataset.Dataset

datasets.bases.read_image(img_path)¶: Keep reading image until succeed. This can avoid IOError incurred by heavy IO process.

datasets.make_dataloader module¶

datasets.make_dataloader.make_dataloader(cfg)¶

# the data augmentation are not carefully modified, other hyper-params may lead to higher performance

uncomment line for other dataset

dataset = CUB(data_dir=cfg.DATA_DIR, verbose=True)

#dataset = UnifiedLoader(dataset_name='Aircraft',data_dir=None,verbose=True)

#dataset = NAbirds(data_dir='/media/space/ZYF/Dataset/Other/NAbirds/', verbose=True)

datasets.make_dataloader.train_collate_fn(batch)¶: collate_fn for training input

datasets.make_dataloader.val_collate_fn(batch)¶: collate_fn for validation input

datasets.preprocessing module¶

class datasets.preprocessing.GaussianMask(probability=0.5)¶: Bases: object

class datasets.preprocessing.RandomErasing(probability=0.5, sl=0.02, sh=0.4, r1=0.3, mean=(0.4914, 0.4822, 0.4465))¶

Bases: object

Randomly selects a rectangle region in an image and erases its pixels.: 'Random Erasing Data Augmentation' by Zhong et al. See https://arxiv.org/pdf/1708.04896.pdf
Args:: probability: The probability that the Random Erasing operation will be performed. sl: Minimum proportion of erased area against input image. sh: Maximum proportion of erased area against input image. r1: Minimum aspect ratio of erased area. mean: Erasing value.

datasets.sampler module¶

class datasets.sampler.ClusterIdentitySampler(data_source, batch_size, num_instances)¶

Bases: torch.utils.data.sampler.Sampler

Randomly sample N identities, then for each identity, randomly sample K instances, therefore batch size is N*K. Args: - data_source (list): list of (img_path, pid, camid). - num_instances (int): number of instances per identity in a batch. - batch_size (int): number of examples in a batch.

class datasets.sampler.RandomIdentitySampler(data_source, batch_size, num_instances)¶