人工智能山水图 | Shan-Shui Paintings by AI

 
0.jpeg

<人工智能山水图> 是我工作室新媒体艺术项目的新系列,这个系列主要目的就是面对当下利用科技流行词(Buzzword)对新媒体艺术作品进行包装鱼龙混杂的市场环境下,真正利用人工智能进行有力的视觉创作,进而呈现真实艺术创作和科学技术之间的关系,让不明觉厉新媒体艺术变得通俗易懂。

<Shan-shui Paintings by AI> is a new series from my studio’s recent new media art projects , this series are focus on how to make a impressive visual creation, present a realistic relationship between the technology and creative art and also shifting the ‘Abstractive Unknowingly Cool’ from new media art to more easy to understand concept  within an overwhelmly decorated by Tech-Buzzword art market.

640.gif

<人工智能山水图>这个系列的技术核心是使用人工智能风格深度学习网络(SytleGAN)学习收集上至两千多张水墨图的像素数据后,利用学习训练后的模型进行自动无限创作模仿山水图样的创作。最后用二维码半吊图案(halftone pattern) 算法转化成可以被扫描的二维码图案。等于创造了一个可以无限制的创作水墨画风的二维码山水动画的机器人🤖️。

<Shan-shui Paintings by AI>  this series’ technical core is using SytleGAN to machine learning over 2000 collective Traditional Eastern Ink painting data, and create a model that can automatically generate Shanshui(Mountains and rivers) Pantings infinitely. And then converted the generated panting content into a scannable QR code pattern with halftone pattern technique. Which in conclude, to create a bot🤖️ that can using QR code to paints ink style Shanshui artwork infinitely.

在互联网收集到的人类手绘水墨画图库Collection data of human ink drawings from internet

在互联网收集到的人类手绘水墨画图库

Collection data of human ink drawings from internet

因为训练一个会模拟风格画图的机器人至少需要至少500到4000张风格参照数据图,所以寻找足够多的采样数据是做这个作品的第一个关键,不过感谢互联网时代#tag标签的流行,我可以快速地利用#水墨画#山水图等等迅速在全网寻找各种各样的图片材料,这其中小到一个初学山水画的小朋友习作照片,大到几百年前古代大师神作扫描数据,可以说是一个全人类关于水墨画意向的一个数据结晶。最后通过我人眼筛辨之后,大约找到了2000张左右的水墨画数据,其中大量的主要是以山水、花草大场景为主的水墨内容。

Because training a bot that simulates style drawing requires at least 500 to 4000 style reference data maps, So finding enough samples is the first key to making this work, thanks to the popularity of the #tag in the Internet era, I can quickly use # 水墨画 # landscapes and so on to quickly find a variety of picture samples on the internet. This range of the samples can start at as amateur as a beginner ’s landscape doodle in photo , or as professional as scanning document of maters’ artwork from hundreds of years ago. Finally, after being handpicked by myself, I collected about 2,000 pieces of ink painting data, which mainly consisted of landscapes, flowers and grass scenes.

互联网收集到的人类手绘水墨画图库Collection data of human ink drawings from internet


互联网收集到的人类手绘水墨画图库

Collection data of human ink drawings from internet

后面的任务实际就是将把这些数据(图片们)交给AI StyleGAN(风格深度学习网络)进行训练和学习,这个过程基本就是等待,我大概等了四个小时后,学习后的模型就基本成形了。当第一次打开学习后的模组绘画的样板后,我还是被眼前的阵列图震住了:

The next task is actually to hand over these data (pictures) to AI StyleGAN for training and learning. This process is basically waiting. After waiting for around four hours, the learned model will basically take shape then. When I first opened the model drawing template, I was pretty shocked by the array of images generated by model in front of me:

人工智能根据训练后的模型自动生成的图库Images array generated by trained AI model

人工智能根据训练后的模型自动生成的图库

Images array generated by trained AI model

正如上图所示,阵列里的每一张图都是人工智能根据训练后的模型自动生成而出的,反正我在看的时候发现每张图都可以看到不同风格、画派、时代以及技法的影子,甚至有时候机器人会模仿落款在画面的合适的位置写一些“火星文书法”。因为模型具有XYZ轴的阵列生成方向,所有每一次自动创作都会沿着XY轴做阵列生成。但是沿着Z轴的变化才是我的需求,虽然Z轴是可以理解成时间无法像XY轴一样直接立马显现出来,但是我可以以序列帧的形式渲染出图最后合成动画,而最后的合成效果着实令我震惊:

As shown in the picture above, each image in the array is automatically generated by the AI based on the trained model. When I looked at it, I found that each picture can see different styles, times, and the technique from traditional ink paintings world, even sometimes the robot will imitate the title and write some "Mars words calligraphy" in the appropriate position on the canvas. Because the model has an array generation direction of XYZ axis, all automatic creation will generate array along the XY axis. But the change along the Z axis is my demand. Although the Z axis can be understood as time but cannot be directly displayed like the XY axis, I can render the picture in the form of sequence frames and finally synthesize the animation, and the final composition really shocked me:

人工智能根据训练后的模型自动生成的动画Animation clip generatated by trained AI model

人工智能根据训练后的模型自动生成的动画

Animation clip generatated by trained AI model

最终沿着Z轴变化生成的序列帧组成动画后,生成的山水图们无缝间地自动延展变化,每个像素仿佛生物变异一样和周围的像素们有序地进行舞动,最后视觉呈现的内容看起来也像几万年来山川河流地貌的变化,我不禁感慨人工智能模拟人类的真TMD可怕。

Finally, after the sequence frames generated along the Z axis change form an animation, the generated landscape images seamlessly automatically extend and change. Each pixel dances in an orderly manner with the surrounding pixels as if it is a biological mutation. The final visual content like mountains and rivers morphing remind me the plate tectonics the past millions of thousands of years on earth. I can't help feeling the potiential f**king power from AI.

将生成画面转化成二维码图案Conversion from Image to QR pattern

将生成画面转化成二维码图案

Conversion from Image to QR pattern

后面部分的工作是将生成的“山水图动画”转移成二维码的图形界面,同时保留了二维码扫描的功能,最后将两个图层合并有机地结合“装裱”起来最终实现了项目的基本架构雏形。目前装裱之后准备了两个方向的版本:

The later part of the work is to transfer the generated "landscape image animation" into the graphical interface of the QR code, while retaining the function of QR code scanning. Finally, the two layers are merged and organically combined to "frame" to finally realize the project The basic structure of the prototype. After mounting, two versions have been prepared:

&lt;人工智能山水图&gt;_2x2阵列|Array_#A1 时长|Duration:8'18"  显示器阵列画框版本|Matrix Displays Frame Version

<人工智能山水图>_2x2阵列|Array_#A1
时长|Duration:8'18"
显示器阵列画框版本|Matrix Displays Frame Version

一个版本可以算是传统架上的版本,这个版本目前的装裱设计是“土豪金”铝合金边框和四个方形显示器组成。这个版本是适用于个人或商务收藏,计划每套视觉限量一个,视觉时长8分18秒(对,非常吉利)。

One version can be regarded as the version on the traditional framing. The current mounting design of this version is composed of a "Rich Gold" aluminum alloy frame and four square displays. This version is suitable for personal or business collections. It is planned that each set of vision is limited to one, and the visual duration is 8 minutes and 18 seconds (yes, very auspicious).

&lt;人工智能山水图&gt;_3x3阵列|Array_#B1 尺寸可变|Variable size 无限镜面沉浸装置版本|Infinite Mirror Immersive Version

<人工智能山水图>_3x3阵列|Array_#B1
尺寸可变|Variable size
无限镜面沉浸装置版本|Infinite Mirror Immersive Version

另一个版本是无限镜面的沉浸装置版本(没错,打卡拍照好看),这个版本由LED面板和四面的镜面反射材料组成,尺寸可变支持定制。沉浸版本通过无限镜面的设置,将人工智能无限生成内容无限可能性和强大的学习能力通过镜面的无限反射寓意“侧面“地向观众表达出来。

The other version is the immersive version with the infinite refelection (yes, perfect for instagram photo feed). This version is composed with an LED panel and four-sided  reflective materials. It is with a variable size supports customization. The immersive version, through the setting of unlimited mirrors, expresses the power of unlimited possibilities of AI and  learning capabilities to the audience on "side "literally .

0 (1).jpeg

关于<人工智能山水图>的这个系列其他方向的版本正在计划和制作当中,其中第一个部分就是会给沉浸装置版本的二维码扫描之后的手机显示的互动内容有一个延伸的探讨。我希望观众拿起手机的时候不仅仅只是拍照打卡,可以通过这样的一个扫描行为有一些更深入的互动探讨

Other versions of this series of <Shan-shui Paintings by AIe> are in the plan and being produced. The first part is an extended discussion of the interactive content displayed on the mobile phone after the scaning of the QR code . I hope that when the audience picks up on their phones, they will not only take pictures and post on social media, but also do some more in-depth interactive discussions through such a scanning behavior.

和我另外一个作品&lt;富春山居图&gt;一样,二维码可以被扫描 Just like another work&lt;Dwelling in the Fuchun Mountains&gt;,QR code can be scanned

和我另外一个作品<富春山居图>一样,二维码可以被扫描
Just like another work<Dwelling in the Fuchun Mountains>,QR code can be scanned

另一个我正在开发合作的版本是AudioVisual声音影像的现场表演版本,在这里也算是一个预告:目前我正在和古琴演奏家,跨界音乐人唐彬老师商讨合作完成一个表演版本的共同作品。下面这个视频是我将唐彬老师2017年的专辑《山水》的音乐片段和沉浸版本合成在一起,让各位先感受一下作品方向的意境和可能性,也算是为这次合作作品展示的一个小预告DEMO:

Another version I am developing and collaborating with is the live performance version of Audio Visual one. For now, I am discussing with Guqin player and cross-border musician Tang Bin to complete a joint version of a performance work. Here is i combine the music  from Tang Bin's 2017 album "Landscape" with the immersive version to intorduce the mood and possibilites of the direction of the work. Thie video below can also be regarded as a small preview DEMO for this collaborative work:

<人工智能山水图>关于这个作品系列更多的未来消息和更新内容,我会继续更新在公众账号的新帖子中,欢迎关注我的公众账号,如果你有更多的问题欢迎留言这个帖子我会及时回复。最后还要声明,本文章内容和文章内作品<人工智能山水图>已经注册著作权和版权保护,受中华人民共和国著作权法、中国版权中心和美利坚合众国著作权法保护,未经曹雨西工作室允许禁止转载。

For more future news and updated content about <Shan-shui Paintings by AI> , I will update the new post in the public account, so please fellow my public account, if you have any questions, please leave a comment below and I will reply in time. Finally, this article and the artwork <Shan-shui Paintings by AI> in this article have been registered with copyright and copyright protection services, protected by the Copyright Law of the People's Republic of China, the Copyright Center of China and the Copyright Law of the United States of America, and are not allowed to be reproduced or reposted without permission from Cao Yuxi Studio.

 

The Making Of ORIENS

 

 

INSPIRATION|灵感起源
 

The initial inspiration were actually start from several posts from NASA on Instagram:
创作起初冲动实际是Instagram上美国航空航天局NASA一套官方图:

This is Cassini, an unmanned spacecraft sent to the planet Saturn, to enter the orbit of the Saturn, and those were picture it took of the Saturn‘s sunlit atmosphere. Also this year we as human being could possibly receive the first real image of the black of our galaxy, The another excited new is this year august 21st, the biggest viewable moon eclipse in 38 years will be able to see in united state land. For me as a visual artist, all this historical moment and perceivable stimulus image trigger my impulse to think about the shapes of this type of sunlit gradient , how this shape could represent the relationship between the individual and the universe context we live in. and how to depict a relationship between the audible perception in time zone and the visual deception in spacial zone. it trigger me to work on my next AudioVisual Installation with the infections of these thinking.

这套图是Cassini探测器在土星和它其他卫星周围拍下的记录,同时在今年的四月欧洲和美国天文局正在开始根据天文望远镜捕捉的银河系数据进行分析开始尝试拼凑出人类认知历史上第一张真实黑洞的照片,不久将会公布于世。同时今年8月21日我们将可以在美国大陆看到38年以来最大壮观的一次日全食。对于我一个视觉艺术家来说,这一系列的历史性时间和感知刺激再次激发了我关于人与宇宙,我们个体与个体所能认知的环境,时间里听觉感知和空间里视觉逻辑等这一系列关系的思考。在这些模糊的灵感信息感染下,我继续了下个关于AudioVisual声音影像装置形式探索和创作。

 

TECHNIQUE|技术探索
 

I didn't choose to use any type of CPU base language tool this time, and want to try some really trendy but really efficiency GPU base Language OpenGL, specially  the shading language of OpenGL(GLSL), big thanks to The Book of Shader by Gonzalez Vivo and Jen Lowe. its really good initial starter book for learning and understand the logic of shader language, and during process of my daily practice of shader, i keep each patch's final visual could render a mood and direction about the visual inspiration i just mentioned above. as you can tell of the screenshot i got here, all these daily creations are the visual base for this new AudioVisual work.

这一次的创作我整体撅起了基于CPU语言种类,尝试了一下相对于视觉艺术家效率更高的GPU架构的语言OpenGL的Shading语言(GLSL), 这里特别提名感谢纽约的Patricio Gonzalez Vivo和Jen Lowe写的一本未完成的网页书叫《The book of shader 》这本书很清晰地入门介绍了shader语言的特征和方法,在整个日常创作中,我一直尝试坚持写出来的内容都是之前介绍的灵感起源的方向。所以整个过程积累了一些图形内容的创作节点的脚本。这些脚本几乎就是Oriens的视觉基础:

Daily Practices of the GLSL Patches in Touchdesigner

For the final platform to integrate this piece into final AV composition, i decided to switch to Touchdesigner this time. For me as a non-traditional software engineer background artist, A node-connect base work flexible interface with a steady solid C++ structure and cross Mac&PC platform is the ideal work environment fir my workflow. which in this case the one and the only TD is my powerful choice. I have to mentioned this special node called Animation , its a keyframe editor in TD, it allow me to publish any type of variables from GLSL patch into timeline. and key-framing is usually the important element for creating audio visual composition, and with this combination of GLSL editor node, i can constantly control all the elements' parameter from my GLSL patch in timeline editing. 

这次承载作品架构平台我最终选择的是TouchDesigner,主要原因是作为非专业程序开发背景出来的艺术家,能通过模块连连看方式,快速实现代码逻辑进行原型系统创作,同时兼顾相对高的稳定性和灵活度,基于C++基础而且还跨PC和MAC平台的TouchDesigner基本是我的首选。这里头格外介绍一些这次作品中核心用到的一个TD的功能节点叫Animation.那这个节点可基于任意的时间轴来发布变量,基本上就是一个实时的AE关键帧编辑器,那对于AudioVisual声音影像装置创作来说是非常重要的一个编辑器。另外要说的就是现在各大软件平台都在逐渐支持的shader编辑器和脚本窗口。TD不算非常好用的但是和Animation配合起来还是一个比较不常见但对于作品创作来说非常有用编辑逻辑:

Animation Node in Touchdesigner interface

 

DESIGN|场域设计
 

About the Final Design of the installation at TodayArt Museum in Beijing, we have a AV projection space that cover all two dimensional walls and floor ,and it enclose it into a interesting mapping space with the big scale of 90feet(30meters) * 45feet(15meters)* 42feet(14meters). you can find out the space on the video below. So in order to create an relative immersive experience for ORIENS flat content, i decide to create another black/inverted colored version of the main content into the floor projection, and due to the view perspective angle from audience to the main wall and floor content, its actually create a passive interaction relationship with the audience on the ground , while they watching it and they are also siting inside of it.
Thanks to the SOP and TOP node from Touchdesigner, i can quickly daft the design with 3d models and uv texture mapping on basic shape, and test them to see it in scale and then do the real model projection test in one software, which is awesome again. this flexible straight forward work process by TD really save me a lot of time.

关于这次装置的空间布局设计,起源于纯单平面的视觉内容,但最终根据今日美术馆的庞大的AV大厅环境,我尝试将立面主屏的内容复制一个阴阳版的地面内容, 因为观赏角度的不一致, 我个人认为还是挺有必要尝试重复相对称的内容,通过布局于地面将观众置身于立面的作品内容中,这样激活一个与观众被动的互动关系。

Design Sketch

Installation Design Preview in Touchdesigner

Mapping Projection Test on 1:100 Models

Real Final Projection In TodayArt Museum in Beijing

 

SOUND|声音与音效
 

On the Sound and Music side, i design the full composition with duration of 6min 38 seconds, and the prelude chapter of the composition is around 2 mins its quite and gentle in order to calm the audience off, the sound effect of prelude section was made by New York Base Musician and Game maker Eddy Wengu Hu, and the main chapter is using the LA base Musician Mars Noir's work :FORCE RECKONED WITH. the rollercoaster emotion curve for lisener from this song composition attract me a lot, and is perfect with the echo we have in this enclosure space in the museum.

ORIENS的整体时间长度大约为6分38秒,跟我上一个作品MACROCSOM的章节排序一样,是前奏章占开头33%主要是安静的低音音效环境加相对肃静的状态让观众进入预备观赏状态,然后使用主章节的66%去完成整个视觉变化呈现,前奏章节的音效是由纽约音乐游戏制作人胡文谷制作的,而主章节的声音我选择的是洛杉矶音乐人Mars Noir的FORCE RECKONED WITH这一首曲子。这首曲子最吸引我的部分是它的节奏和音效风格非常末世DOOM的感觉,同时有大量的回音和回响充斥其中,相对可以利用今日美术馆AV大厅的封闭但不吸音的空间进行更多的回响混合。

The SoundCloud links of those two musicians
两位音乐人SoundCloud链接:
Mars Noir:https://soundcloud.com/marsnoir
胡文谷:https://soundcloud.com/dgspitzer

 

FINAL|最终呈现

 

The contents of the final device through the real scale of the AV hall presented when the first time I see the person is still very excited, after all, a huge round object to the dynamic form of my design with the sound of the time in the space, The whole device expressed on the relationship between man and the universe can finally be in such a perfect fit among the body, of course, I hope that the conditions of the audience in the few people standing or lying on the ground center of such a best view Angle to feel this work:

最终装置的内容通过AV大厅真实尺度呈现出来的时候,第一次看到时候我个人还是很激动的,毕竟一个硕大的圆形物体以我设计的动态形式配合声音出现的空间之中的时候,整个装置表达出来的关于人与宇宙的形态关系终于可以在这样的一个完美体量当中呈现,当然我跟希望有条件的观众在人少的时候站在或躺着地面的中心这样一个最佳观赏角度去感受这个作品:

Photo by Show Sponsor XiaoMi

Here is the version 1.26 photo doc on site:

Here is the version 1.26 view in the 4th Floor on site:

EXHIBITION INFORMATION|展览信息
 

If you are in Beijing, you are so welcome to jump in and check the show at TodayArtMuseum.
Here is the detail of the show:

Name: zip 未来的狂想 | 小米∙今日未来馆
Time 展览时间:07/16- 09/16 | 2017
Space 展览场地:今日美术馆1号馆

Address:今日美术馆/Today Art Museum
北京市朝阳区百子湾路32号,100022
No.32 Baiziwan Road, Chaoyang District, Beijing, China,100022
www.todayartmuseum.com

 

 

DailyShader_008

GLSL Gallery