Update (November 20, 2020)
On November 17, 2020, MN-3 was certified as number one among the Green500 systems with highest-quality Level 3 measurement.

TOKYO – November 17, 2020 – Preferred Networks, Inc. (PFN) and Kobe University announced today that MN-3, PFN’s deep learning supercomputer, has achieved an energy efficiency of 26.04 gigaflops-per-watt (Gflops/W), 23.3% above its previous record that topped the Green500 list of the world’s most energy-efficient supercomputers in June 2020. With this new achievement, MN-3 ranked 2nd in the latest Green500 list for November 2020.

PFN’s MN-3 deep learning supercomputer

Powered by MN-Core, a highly efficient custom processor co-developed by PFN and Kobe University specifically for use in deep learning, MN-3 started operation in May 2020 on a trial basis. Drawing on its software development expertise, PFN continuously improved MN-3’s software stack, especially for command processing and data transfer between memory units, which resulted in a higher computing performance with less nodes and processors than in June 2020.

PFN plans to continue developing software for MN-3 to use the supercomputer for its internal research and development purposes, including autonomous driving, robotics and drug discovery. 

The comparison of systems used for measurement and their respective performance are as follows.

https://www.top500.org/system/179806/

Note: The TOP500 entry states that MN-3 has 1,664 cores. This number consists of 128 MN-Core processors, counted as one core each, and 1,536 Intel Xeon processors. MN-Core performs most of the computations for the HPL benchmark measurement.

投稿 Preferred Networks’ MN-3 Supercomputer Breaks Previous Record by 23.3% は Preferred Networks, Inc. に最初に表示されました。

PFN’s deep learning-based image analysis tool indicates chest x-ray abnormalities

(The images above may differ from the actual tool)

The diagnostic assistance tool uses a model based on PFN’s unique deep learning algorithms and was trained with a large number of actual chest X-ray images paired with lung cancer diagnosis. The model analyzes the test takers’ chest X-ray images and automatically indicates abnormalities that may represent lung cancer, which is expected to allow physicians to interpret them quickly and accurately. During the trial, two physicians will interpret each chest X-ray image as recommended by Japan’s Ministry of Health, Labour and Welfare (MHLW) while they refer to the analysis results. The test data will be securely managed on NOBORI, Nobori’s cloud-based platform that allows medical institutions to store and use anonymized medical information.

PFN demonstrated its medical image analysis technology when the company ranked sixth out of 1,499 teams from around the world at Kaggle RSNA Pneumonia Detection Challenge co-hosted by Kaggle and Radiological Society of North America in 2018, in which the teams competed on accuracy to detect potential pneumonia cases from chest X-ray images.

The shortage and workload of physicians who are able to interpret numerous X-ray images each day are known issues in Japan. Although X-ray interpretation requires extensive training and experience for physicians, it is still the most common method, compared to the more expensive and lengthier alternatives. According to the Japan’s National Cancer Center statistics, lung cancer had the highest cancer mortality rate for males and second highest for females in 2018. The top cause of death in Japan in 2018 was cancer according to MHLW’s data.

投稿 Preferred Networks Uses Deep Learning to Help Kyoto Physicians Diagnose Lung Cancer from Chest X-Ray Images は Preferred Networks, Inc. に最初に表示されました。

CuPy v8 provides the following new features:

• Support for CUDA 11 and the latest NVIDIA GPU (Ampere architecture)
  Boosts single-precision mathematics using TensorFloat-32 (TF32) computation mode
• Official support for NVIDIA cuTENSOR/CUB
  Performance improvements up to 9.7x for matrix computations in our benchmarks (see blog post for details)
• Enhanced kernel fusion
  Now supports merging computations including multiple reductions into a single kernel
• Automatic tuning of kernel launch parameters using Optuna
  Discover the optimal launch parameters depending on the data properties to improve performance
• Memory pool sharing with external libraries
  Improved interoperability with PyTorch by using pytorch-pfn-extras; for example, you can flexibly integrate CuPy as a preprocess code into the PyTorch workflow
• Improved NumPy/SciPy function coverage
  Many functions added, including the NumPy Polynomials package (results of Google Summer of Code 2020) and the SciPy image processing package

PFN will continue to swiftly incorporate the latest research outcomes while collaborating with supporting companies and open source communities for the development of CuPy.

投稿 Preferred Networks Releases CuPy v8 は Preferred Networks, Inc. に最初に表示されました。

The Tokyo-based joint venture aims to apply deep learning technology to seismic analysis, a common method to find underground oil and gas reservoirs using artificially induced shock waves. By using PFN’s supercomputer for large-scale simulation of seismic wave propagation, Mit-PFN Energy aims to develop a solution that accurately estimates geological structure for efficient utilization of subsurface resources.

In addition to oil and gas, the new joint venture also plans to use its solution for subsurface carbon capture and storage (CCS) as well as renewable energy including geothermal.

Mit-PFN Energy is 51 percent owned by Mitsui and 49 percent by PFN, and headed by Haruaki Moritani.

Going forward, PFN and Mitsui will continue to build business models using deep learning technology while collaborating with partner companies and utilizing the Mitsui & Co. Group’s wide-ranging business assets.

投稿 Preferred Networks and Mitsui Establish Joint Venture for Deep Learning-Based Subsurface Structure Analysis は Preferred Networks, Inc. に最初に表示されました。

PFRL implements a comprehensive set of DRL algorithms and techniques drawn from the state-of-the-art research in the field, allowing researchers to quickly compare, combine and experiment with them for fast iteration. In particular, PFRL offers high-quality, thoroughly benchmarked implementations attempting to reproduce nine key DRL algorithms which can be used as a base for research and development. By using PFRL, existing ChainerRL users will be able to retain much of their existing code as they migrate to PyTorch.

PFN will provide baseline implementations using PFRL for the MineRL competition at the 2020 Conference on Neural Information Processing Systems (NeurIPS), which participants can use as a starting point to develop their own novel systems. PFRL also provides examples for using Optuna to enable hyperparameter search for DRL applications.

PFN released PFRL in response to the PyTorch community’s demand for a comprehensive DRL library similar to ChainerRL. First launched in February 2017, ChainerRL has been applied internally at PFN and in the Chainer community outside the company in a variety of research and industrial settings. Going forward, PFN aims to use PFRL to accelerate internal research and development, as well as to serve the broader reinforcement learning community.

PFRL is currently available at: https://github.com/pfnet/pfrl

投稿 Preferred Networks Releases PFRL Deep Reinforcement Learning Library for PyTorch Users は Preferred Networks, Inc. に最初に表示されました。

Hyperparameter importance graphic, showing which hyperparameters in a neural network matter most

Optuna v2.0 has the following new features:

• Hyperparameter importance evaluation

Optuna can provide feedback on how important each of the measured hyperparameters were on the overall performance of the algorithm. This valuable information can assist researchers and developers to focus on tuning the hyperparameters that matter most.

• Hyperband pruning

Pruning allows unpromising trials to be stopped early. One of the most recent and robust techniques for pruning is Hyperband, which is well-suited for deep learning and is now available in Optuna.

• Performance improvements

Optimization has been speeded up by improving the lower storage layer. Experiments show that searches can be up to ten times faster.

• Additional integrations

Additional integration modules are available for easy use with LightGBM to do efficient stepwise optimization as well as MLflow, AllenNLP, and TensorBoard.

Since open-sourced in December 2018, the interest from researchers and developers for Optuna has grown. In the last month, Optuna was downloaded over 100,000 times. Going forward, PFN plans to work on multi-objective optimization to allow multiple criteria to be optimized simultaneously, along with continuing to add integrations and improve the performance of Optuna.

About Optuna
Optuna was open-sourced by PFN in December 2018 as a hyperparameter optimization framework written in Python. Optuna automates the trial-and-error process of finding hyperparameters that deliver good performance. Optuna is used in many PFN projects and was an important factor in PFDet team’s award-winning performances in the first Kaggle Open Images object detection competition. https://optuna.org/

投稿 Preferred Networks Releases Optuna v2.0 は Preferred Networks, Inc. に最初に表示されました。

As the first product in the new business domain, PFN has developed Playgram, a programming education app primarily targeting students in elementary school and above. PFN has teamed up with Yaruki Switch Group (YSG), Japan’s leading education group with a diverse range of programs and over 1,700 schools, to build a programming course package using Playgram. Beginning August 2020, YSG will first pilot the package in three schools in the Tokyo area, both in classrooms and online.

Developed by PFN’s software engineers at the forefront of artificial intelligence technologies, Playgram incorporates the K-12 Computer Science Framework, a U.S. guideline for computer science education. The app will be available in Japanese at launch.

Playgram is characterized by the following features:

1. Bridges the gap between visual and text-based coding
   Children can learn how to program step by step at their own pace, starting from block-based visual programming, then moving on to typing and text-based coding using Python.
2. Rich 3D interface that inspires creativity
   By moving robots and flying a character in the sky in a video game-like 3D world, children can enjoy using programming to solve problems while enhancing creativity and spatial reasoning.
3. Adaptive learning system and user-friendly tutorials
   Playgram analyzes learning data to identify the child’s strengths and weaknesses, visualizes the progress and personalizes the learning experience, which helps teachers and parents give instructions even without programming knowledge.

Playgram allows children to learn visual programming first and then move on to text-based coding

By combining Playgram’s fun, creativity-inspiring content and YSG’s proven educational method that nurtures motivation, PFN and YSG aim to foster children’s problem-solving abilities and creativity. PFN plans to develop Playgram into a platform that opens up learning opportunities for new technologies including augmented reality, the internet of things and artificial intelligence.

Toru Nishikawa, Chief Executive Officer of PFN, said:

“Computers are evolving, and we must see it as an opportunity, not a threat. For the new generation to see it this way as they grow up, it is essential for us to provide them with ample opportunities now to familiarize themselves with programming at early ages and develop logical and creative thinking. As a company that breathes the latest technologies in computer science on a daily basis, PFN is in the right position to teach them how fun it is to develop programs to solve problems. We look forward to working with Yaruki Switch Group to leverage their proven pedagogical method to enable all children to acquire programming skills.”

PFN has previously published free online materials to promote the use of deep learning and other artificial intelligence technologies, as well as to foster accurate understanding. By launching a for-profit education business, PFN expects to gain additional resources to develop high-quality content. Going forward, PFN aims to provide additional opportunities for people of all age groups, from elementary school children to professionals, to acquire computer science knowledge and skills.

投稿 Preferred Networks to Launch Computer Science Education Business は Preferred Networks, Inc. に最初に表示されました。

MN-Core

PFN’s MN-3 deep learning supercomputer

MN-3 has achieved an energy efficiency of 21.11 gigaflops-per-watt (Gflops/W) based on the industry’s standard high-performance Linpack (HPL) benchmark, meaning it performed 21.11 billion calculations for every watt of power consumed in one second. The achievement is 15% higher than the previous Green500 record of 18.404 Gflops/W, which was recorded in June 2018. This demonstrates that MN-Core and MN-3 are leading the global competition for energy-efficient supercomputers specialized for deep learning.

MN-3, which is located in Japan Agency for Marine-Earth Science and Technology (JAMSTEC)’s Simulator Building at Yokohama Institute for Earth Sciences, started operation in May 2020. The system used for MN-3’s performance measurement consisted of 40 nodes and 160 MN-Core processors.

• Peak performance (theoretical): 3.92 Pflops
• Speed of solving linear simultaneous equations (HPL Benchmark): 1.62 Pflops
• Performance for every watt of power consumed: 21.11 Gflops/W

https://www.top500.org/system/179806/

Note: The TOP500 entry states that MN-3 has 2,080 cores. This number consists of 160 MN-Core processors, counted as one core each, and 1,920 Intel Xeon processors. MN-Core performs most of the computations for the HPL benchmark measurement.

The key elements that contributed to the achievement are as follows.

1. MN-Core
MN-Core, developed by PFN and Kobe University with support from RIKEN AICS/R-CCS, is equipped with highly efficient compute units designed specifically for deep learning.

MN-Core board

2. MN-Core DirectConnect
The MN-Core DirectConnect interconnect facilitates high-speed, high-efficiency data transmission between the nodes.

3. Optimization techniques for efficient workload management
The performance optimizations used in the HPL Benchmark can also be used to speed up deep learning workloads.

4. High compute-density and locality
The energy efficiency was maximized by densely integrating multiple MN-Core dies onto each board.

The technologies that drastically reduced the environmental impact and operation costs are expected to become a foundation for highly efficient information systems in general as well as supercomputers of the next generation.

PFN plans to further increase MN-3’s energy efficiency by improving the installation methods, cooling and MN-Core-specific middleware.

For more information about PFN’s supercomputers, visit: https://projects.preferred.jp/supercomputers/en/

投稿 Preferred Networks’ MN-3 Tops Green500 List of World’s Most Energy-Efficient Supercomputers は Preferred Networks, Inc. に最初に表示されました。

PFN’s mobile manipulator has been selected as one of the projects to receive subsidies from Japan’s Ministry of Economy, Trade and Industry under its 2020 program to support Japanese startups combining software and hardware to open up new markets. From June 2020 to February 2021, PFN will use the subsidy to design the mobile manipulator’s hardware and perform field tests in various real-world settings.

Concept rendering of PFN’s mobile manipulator

PFN takes a vertically-integrated approach by developing the software, including the motor control layer that regulates physical interaction and safety, and designing the hardware, such as the arm, end effectors and mobile platform. By developing both hardware and software in-house, PFN can exclude unnecessary functions, minimize production costs, and enhance the product’s safety and performance.

Since announcing its “Robots for Everyone” vision in 2018, PFN has devoted itself to developing robots that can be installed at a low cost in various environments and support humans with their everyday needs. PFN will continue combining hardware technologies with its original strength in software technologies, such as deep learning for object recognition and control, to create practical robots that adapt flexibly in varying real-world conditions.

PFN is looking for partners and facilities that can participate in the field tests in Japan. Inquiries and suggestions are accepted via the contact form:
https://forms.gle/Sx3k4oSn71gowuVE9

投稿 Preferred Networks Begins Designing Mobile Manipulators for Mass Production は Preferred Networks, Inc. に最初に表示されました。

The pytorch-pfn-extras library includes several popular Chainer functions from user feedback during PFN’s transition from the Chainer deep learning framework to PyTorch.

pytorch-pfn-extras includes the following features:

• Extensions and reporter

Functions frequently used when implementing deep learning training programs, such as collecting metrics during training and visualizing training progress

• Automatic inference of parameter sizes

Easier network definitions by automatically inferring the sizes of linear or convolution layer parameters via input sizes

• Distributed snapshots

Reduce the costs of implementing distributed deep learning with automated backup, loading, and generation management of snapshots

pytorch-pfn-extras is available at: https://github.com/pfnet/pytorch-pfn-extras 

The migration guide from Chainer to PyTorch can also be found at: https://medium.com/pytorch/migration-from-chainer-to-pytorch-8ed92c12c8 

On April 6, Optuna was added to the PyTorch Ecosystem of tools that are officially endorsed by the PyTorch community for use in PyTorch-based machine learning and deep learning research and development.

PFN is discussing merging pytorch-pfn-extras features into the PyTorch base build with the PyTorch development team at Facebook, Inc. In response to strong demand from both internal and external users, PFN also aims to release a PyTorch version of the deep reinforcement learning library, ChainerRL, as open-source software by the end of June 2020.  

PFN aims to continue leveraging its software technology it has accumulated through the development of Chainer to contribute to the development of PyTorch and the open-source community.

The PyTorch team at Facebook commented:

“We appreciate PFN for contributing important Chainer functions, such as gathering metrics and managing distributed snapshots, through pytorch-pfn-extras. With this newly available library, PyTorch developers have the ability to understand their model performances and optimize training costs. We look forward to continued collaboration with PFN to bring more contributions to the community, like ChainerRL capabilities later this summer.”

投稿 Preferred Networks Deepens Collaboration with PyTorch Community は Preferred Networks, Inc. に最初に表示されました。